(19) 



J 



Europaisches Patentamt 
European Patent Office 
Office europeen des brevets 




(12) 



(43) Date of publication: 

07.06.2000 Bulletin 2000/23 



(n) EP 1 006 710 A2 

EUROPEAN PATENT APPLICATION 

(51) Intel/: H04N 1/32 



(21) Application number: 99309490.3 

(22) Date of filing: 29.11.1999 



(84) Designated Contracting States: 

AT BE CH CY DE DK ES Fl FR GB GR IE IT LI LU 
MC NL PT SE 

Designated Extension States: 
ALLTLVMKROSI 

(30) Priority: 30.11.1998 JP 34001998 

(71) Applicant: KABUSHIKI KAISHA TOSHIBA 
Kawasaki-shi, Kanagawa-ken 210-8520 (JP) 



(72) Inventors: 

• Muratani, Hirof umi, c/o Intel. Property Dlv. 
Tokyo 105-8001 (JP) 

• Katoh, Taku, c/o Intel. Property Div. 
Tokyo 105-8001 (JP) 

• Endo, Naoki, Intel. Property Div. 
Tokyo 105-8001 (JP) 

(74) Representative: Shindler, Nigel 
BATCHELLOR, KIRK & CO. 
102-108 Cler ken well Road 
London EC1M 5SA(GB) 



(54) Digital watermark embedding and detection devices 



(57) A digital watermark embedding device has an 
input section which inputs pixel value data of an image 
which is a target for embedding ol watermark informa- 
tion, an embedded position selection section which de- 
termines a position in frequency domain into which the 
watermark information is to be embedded, irrespectively 
of a frequency component value, a frequency compo- 
nent value variation decision section which determines 
a variation of a frequency component value at each em- 
bedded position determined by the embedded position 
selection section, a pixel value variation calculation sec- 
tion which calculates a variation of a pixel value at each 



point in pixel value domain of the image based on the 
embedded position in frequency domain determined by 
the smbedded position sBlection section and on the var- 
iation of a frequency component value determined by 
the frequency component value variation decision sec- 
lion, a pixel value conversion section which embeds wa- 
termark information by changing corresponding pixel 
value data of the image in accordance with the variation 
of a pixel value at each point in pixel value domain cal- 
culated by the pixel value variation calculation section, 
and an output section which outputs pixel value data of 
the image into which watermark information has been 
embedded. 
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Description 

[0001] The present invention relates to a digital watermark embedding device and a digital watermark detection 
device suited to protect a copyright and to control use oi data on digital literary properties represented by images and 
the like by embedding into the data, information (to be referred to as 'watermark information' hereinafter), such as 
discrimination information on copyright owners and users of the data, information on the right of the copyright owners, 
conditions for using the data and confidential information necessary to use the dala, in a stale in which the watermark 
information is not easily perceived and then by detecting corresponding watermark information from dala into which 
the watermark information is embedded al need. 

[0002] In recent years, there is a growing interest in the copyright of digital literary properties such as an image (a 
static image, a dynamic image), voice data, music data, design data, CG (computer graphic) data and text data. As 
one of techniques for protecting the digital literary properties, there ia known a digital watermark technique. 
[0003] The digital watermark technique is mainly aimed 1o protect the copyright of the data and control use thereof 
by embedding, i.e., eoncealingly inputting into a digital literary property, watermark information ('WW information) such 
as discrimination information on the copyright owner and users of the data, information on the right of the copyright 
owner, conditions for using the data and confidential information necessary to use the data, in a state in which the 
watermark information cannot be easily perceived. 

[0004] For the purpose of this type, the digital watermark technique is required to satisfy, in particular, the following 
characteristics: 

(1) Robustness characteristics 

Watermark information is difficult to remove from a literary property and to lose even by various data operations 
(filtering process, data compression/ expansion, D/A conversion, A/'D conversion and the like). If data is intention- 
ally manipulated to remove the watermark information, data quality greatly deteriorates. 

(2) Perceptual invisibility characteristics 

The literary property into which watermark information is embedded is perceived similarly as the original literary 
property. 



[0005] The digital watermark technique is also required to have embedding efficiency for efficiently embedding wa- 
30 termark information, detection efficiency lor efficiently detecting watermark information and the like. 

[0006] Now, as a typical method of embedding watermark information into, for example, image data by means of the 
digital watermark technique, there is known a method of converting image data into frequency domains and then 
embedding watermark information to a specific frequency component. 

[0007] FIG. 1 is a block diagram showing the constitution oi a conventional digital watermark embedding device 200 

35 for embedding watermark information Into the image dala. 

[0008] In the device 200, orthogonal transform, such as DFT (discrete Fourier transform), DCT (discrete cosine 
■ transform), wavelet transform or Hadamard transform, is conducted to image dala in pixel value domain which is a 
target for embedding of watermark information, by an orthogonal transform section 201 and frequency component 
value data forming image data in frequency domain is obtained. 

40 [0009] Next, watermark (Wfvt) information is embedded into a specific frequency component in the image data in 
frequency domain in the form of noise having a peak by a digital watermark embedding section 202. The watermark 
information embedding is conducted in the digital watermark embedding section 202 as follows. 
[0010] First, based on the frequency component data forming the image data in frequency domain, an embedding 
frequency component selection section 202a selects a frequency component suited for embedding of watermark in- 

15 formation, e.g., a frequency component having a peak in case of embedding the above-stated watermark information 
as noise. The frequency component variation decision seclion 202b determines the variation of this frequency com- 
ponent value. The frequency component change, section 202c changes the frequency component value forming the 
image data In frequency domain by a variation determined by the frequency component variation decision section 202b 
and thereby generates frequency component value data on the image into which the watermark information has been 

so embedded. 

[0011] Next, inverse transform of the orthogonal transform which has been conducted by the orthogonal transform 
section 201 , is conducted to the image data in frequency damam, into which watermark information has been embedded 
by the digital watermark embedding section 202, by a reverse-orthogonal Iransform section 203, to thereby obtain 
image data in pixel value domain into which watermark information has been embedded. 

[0012] Thus, the walermark infoimation embedded by the digital watermark device 200 shown in FIG. 1 is embedded 
into the image dala in pixel value domain in a slate in which the watermark inlormation is dispersed to the entire image 
and, therefore, robust against various operations. In addition, if the frequency component into which watermark infor- 
mation is embedded is the frequency component of an intermediate frequency or higher, the walermark information is 
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not lost even by a low frequency pass filter which causes the loss of the LSB information of Ihe' image data. 
[0013] With the constitution shown in FIG. 1, since it is necessary to perform orthogonal transformation twice by 
means of orthogonal transform, heavy load processing is required. Normally, therefore, an image is first divided into a 
plurality of blocks. For example, it is assumed that the number of pixels of a certain image in horizontal direction and 

£ vertical direction are W and H, respectively. The image is divided into a plurality of rectangular blocks each of which 
includes w and h pixels in horizontal and vertical directions, respectively. While a pixel positioned on upper-left end 
point in the image is set as an origin, the number of pixels is counted, if a pixel is in the x-th row and the y-th column, 
the position of the pixel is expressed as (x, y). If a certain pixel satisfies the following relationships: i X w =s x < (i + 
1) X w and hxh£y<fj+i)xh, then it belongs to a block positioned in the i-th row in horizontal direction and the 

to j-th column in vertical direction. All of or part of the blocks obtained by dividing the image are subjected to orthogonal 
transformation, respectively and watermark informal ton is embedded into the blocks. 

[0014] If orthogonal transform is used to embed watermark information, the quantity of embedding operation is the 
order of the square of a processed block size. As the block size is larger, the operation quantity considerably increases. 
Due to this, dedicated hardware is needed if watermark information is embedded into, for example, a dynamic image 
is [0015] FIG. 2 is a block diagram showing 1he constitution of a conventional digital watermark detection device 210 
for detecting corresponding watermark information from the image data into which watermark information has been 
embedded by the digital watermark embedding device 200 shown in FIG. 1 . 

[0016] In the device 201, image data in frequency domain is converted to frequency component value data forming 
image data in frequency domain by orthogonally transforming the image data in pixel value domain into which watermark 
so information has been embedded by an orthogonal transfo.-m section 211 . Next, a digital watermark detection section 
212 detects watermark information having a peak in a specific frequency component from the frequency component 
value data. 

[0017] To embed watermark information into target data so that the watermark information is not likely to be bst by 
various operations, there is proposed a method for spreading and embedding watermark information into a wide range 

25 of data by the spread spectrum technique. 

[0018] FIG 3 is a block diagram showing the constitution of a conventional digital watermark embedding device 220 
for embedding image data while using spread spectrum of PN (pseudo-random number) sequence. 
[0019] In the device 220. a pixel value at each point in pixel value domain of an image which isatarget for embedding 
of watermark information, is subjected to direct spread spectrum (SS) technique, i.e., the pixel value is multiplied by 

30 the pseudo-random number of {-1,1} referred to as PN sequence generated by a PN sequence generation section 
220F, by a spread spectrum section 220A. Then, orlhogonal transform, such as DCT transform, is conducted to the 
pixel value at each point in pixel value domain, to which the spread spectrum is conducted, by an orthogonal transfor- 
mation section 220B, thereby obtaining Irequency component value dala forming the spread spectrum image data in 
frequency domain. 

35 [0020] Next, a digital watermark embedding section 220C embeds watermark information in the form of noise having 
a peak in a specific frequency component in the spread spectrum image data in Irequency domain. Then, inverse 
transform of the orthogonal transform previously conducted by the orthogonal transform section 220B is conducted to 
the spread spectrum image data in frequency domain into which the watermark information has been embedded, by 
a reverse-orthogonal transformation section 220D, thereby ootaining spread spectrum imago data in pixel value domain 

40 into which the watermark information has been embedded. Thereafter, inverse spectium spreading of the spread spec- 
trum previously conducted by the spread spectrum section 220A is conducted to the spread spectrum image data in 
pixel value domain into which the watermark information has been embedded, by an inverse spectrum spreading 
section 220E, thereby obtaining image data in pixel value domain into which the watermark information has been 
embedded. 

is [0021] As stated above, according to the method for conducting spread spectrum using PN sequence and orthogonal 
transform and then embedding watermark information into target data, the frequency component into which watermark 
information is embedded may not be necessarily an intermediate frequency component since spread spectrum using 
PN sequence is conducted. If watermark information is embedded in the form of a peak, inverse spectrum spreading 
has an effect of thinly dispersing the watermark information into a wide range of frequency components. 

so [0022] The watermark information embedding as staled above may be possibly effective against data operation by 
A/D (analog/digital) conversion, D/A (digital/analog) conversion and data compression and expansion, as well. How- 
ever, since A/D conversion, D/A conversion and data compression and expansion have a great effect on image data, 
watermark information embedding is desirably conducted by spreading the information into as wide a range as possible. 
[0023] FIG. 4 is a block diagram showing the constitution of a conventional digital watermark detection section 230 

ss for detecting corresponding watermark information from the image data into which watermark information has been 
embedded by the digital watermark embedding device 220 shown in FIG. 3. 

[0024] In the device 230, spread spectrum is conducted to the image data in pixel value domain, into which watermark 
information has been embedded, by a spread spectrum section 231 , using the PN sequence generated by a PN se- 
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quence generation section 234. Then, orthogonal transform, such as DCT transform, is conducted to a pixel value at 
each point in pixel value domain 1o which the spread spectrum has been conducted, by an orthogonal transformation 
section 232, and frequency component data forming spread spectrum image data in frequency domain is thereby 
obtained. Thereafter, walermark information having a peak in a specific frequency component is detected from the 
obtained frequency component value data by a digital watermark detection section 233. 

[0025] Further, a technique for embedding watermark information by a patchwork method has been conventionally 
proposed. FIG. 5 shows the blockdiagram of a conventional digital watermark embedding device employing this tech- 
nique. 

[0026] In a digital watermark embedding device 240 shown in FIG. 5, two pixels are first selected as a pair<(ax, ay), 
(bx, by)> from the pixel value data on an image into which is a target for embedding of walermark information, by a 
pixel value pair generation section 241 . 

[0027] Next, if 1 is embedded as watermark information, tor example, the pixel value a of the first component in the 
pair, i.e., the first component (ax, ay) is incremented by 1 by the first component change section 242, whereas the pixel 
value b of Ihe second component in the pair, i.e., the second componenl (bx, by) is decremented by 1 by the second 
component change section 243. if 0 is embedded as watermark information, by contrast, the pixel value of the first 
component is decremented by 1 by the first component change section 242, whereas the pixel value of the second 
component is incremented by 1 by the second component change section 243. The above-stated operation is con- 
ducted to a plurality of pairs selected at random so as not to overlap each olher by Ihe pixel value pair generation 
section 241 . 

[0028] As a result, if the number of pixels is sufficiently high, the statistically significant difference is expected between 
the distribution of pixel values of the first components in respective pairs and that of the pixel values of the second 
components in the respective pairs, as shown in FIG. 6. 

[0D29J FIG. 7 is a block diagram showing the constitution of a conventional digital watermark detection device for 
detecting corresponding watermark information from the image data into which walermark information has been em- 
bedded by the digital watermark embedding device 240 shown in FIG. 5 by means of a patchwork method. 
[0030] In a digital watermark detectbn device 250 shown in FIG. 7, the same pair of pixels as those generated by 
the pixel value pair generation section 241 in the digital watermark embedding device 250 shown in FIG. 5, are gen- 
erated by a pixel value pair generation section 261 . The average of pixel values in the first component of the respective 
pairs is calculated by the first component average calculation section 2S? and the average of pixel values in the second 
component of the respective pixels is calculated by the second component average calculation section 263. 
[0031] Next, in an average comparison section 264, the two average values are comparedand embedded watermark 
information is detected (recovered) by judging that 1 is embedded if the averageof the pixel values in ihefirst component 
is higher, and that 0 is embedded if the average of the pixels values in the second component is higher. 
[0032] The above-stated conventional digital embedding anddetection techniques have advantage in that walermark 
information is difficult to lose by various operations. On the other hand, these techniques, which require orthogonal 
transform such as DCT transform, have disadvantage in that complexity of computation for embedding and delecting 
watermark information and a hardware cost are large. That is, DCT transform is a kind of transform introduced by 
transforming form DFT and the complexity of computation is the same order. A processing of DFT can be performed 
with high speed by FFT algorithm, and, theoretically, the complexity of computation of the FFT for data of data size N 
is given as Q (N X log N). 

[0033] Complexity of computation is roughly evaluated using, for example, a static image ol W wide by H high. If this 
static image is divided into blocks of w wide by h high, the complexity of computation per block for the DCT transform 
is given as O (h x w x log w + w X h x log h) = O (w x hx log (w x h)). While the total number of blocks is (VWw)x 
(H/h), the total complexity of computation is: 

0((W/w) X (H/h) X w X h X log {w x h)) 
= O (W x H x log (w x h)). 

[0034] The calculation ratio of a case where an image is a block per se to a case where an image is divided into 
blocks of w wide by h high is: 

(log(WxH))/(log(wXh)). 
[003S] Here, it w = w x 21 and H = h X 2 r , then 
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<log(Wx H))/(log(wX h)). 
= (1 + (p + q)/log 2 (wXh). 

5 

[0036] For example, if W = H = 1024 = 10 24 and w = h = 32 = 2 5 , then q = r = 5 and (1 + (p 4- q)/log 2 (wxh) = 2. It 
follows thai the processing speed at which an image is divided into small blocks each having a side which is 1/32 as 
small as that of an image and DOT transform is conducted to the small blocks is about twice as fast as that at which 
DCT transform is conducted to an entire image. 
10 [0037] However, since the hardware cost is large to mount FFT to a large size of the block with hardware, actually, 
the hardware mounting is often adapted to perform processing by repeatedly using small FFT. The complexity of com- 
putation in this case still becomes the order ol ths second power of the data size. 

[0038] How wide the region into which watermark information can be dispersed and embedded will be depends on 
the complexity of computation of orthogonal transform such as DCT transform and time allowed to embed and detect 
« the watermark information. If it is possible to lake sufficient time to embed and detect watermark information, efficiency 
is not an important factor. From the viewpoint of robustness characteristics, if embedding and detection time falls within 
a realistic range, it is desirable to set the size of a block to be as large as possible. 

[0039] In case of using watermark information lor the control of use and rating, however, it is impossible to take a 
lot of time to detect watermark information. In case of using watermark information for finger print, it is impossible to 

20 take a lot of time to embed watermark information, either. 

[0040] As stated above, according to the conventional digital watermark embedding and detection technique, in spite 
of the fact that it is desirable to select a large block, size (such as 512 x 51 2 pixels and 720 X 720 pixels) and thinly 
and widely disperse watermark information in view ol robustness characteristics and perceived non-revealment char- 
acteristics, processing for embedding and detecting watermark information cannot be efficiently conducted due to the 

25 problem of cost required for the calculation for orthogonal transform. As a result, the conventional lechnique is disad- 
vantageous in that there is no other choice but to select a small block size (such as 16 x 16 pixels or 32 X 32 pixels) 
and to embed watermark information into the small blocks in case of application requiring high efficiency in the em- 
bedding and delection of watermark information. 

[0041] Meanwhile, according to the conventional digital watermark embedding and detection technique by means 
30 of the patchwork method, although it is not required to conduct orthogonal transform such as DCT transform, image 
data into which watermark information is embedded does not have statistically uniform property. Owing to this, if the 
number of pixel pairs selected (generated) at random is small, no basis is formed on which it is judged whether the 
difference in statistical distribution between the first component and the second component in the pairs is due to the 
embedding operation or a failure to generate sufficient number of pixel pairs, for which reason statistical difference 
35 originally exists between the pairs selected by chance. As a result, Ihe conventional technique is disadvantageous in 
that it requires sufficient number of pairs to make the statistically significant difference in distribution between the first 
and second components in the respective pairs after embedding operation and in that it is difficult to estimate the 
necessary number of pixel pairs to be generated during embedding and detection in advance. 
[0042] It is, therefore, an object of the present inven-ion to provide a digital watermark embedding device capable 
40 of efficiently embedding watermark information into target data even if a large block size is selected to divide the data. 
[0043] It Is another object ofthe present invention to provide a digital watermark detection device capable of efficiently 
detecting embedded watermark information even if targe block size of information is selected or an entire image per 
se is treated as one block. 

[0044] It is yet another object ofthe present invention Id provide a digital watermark embedding device and a digital 
4$ watermark detection device capable of estimating the necessary number of pixel pairs to be generated while watermark 
information is being embedded and detected by employing a patchwork melhod. 

[0045] A digital watermark information according to the present invention is characterized by comprising an input 
section which inputs pixel value data of an image which is a target for embedding of watermark information; an em- 
bedded position selection section which determines a position in frequency domain into which the watermark informa- 

so tion is to be embedded, irrespectively of a frequency component value; a frequency component value variation decision 
section which determines a variation of a frequency component value at each embedded position determined by the 
embedded position selection section; a pixel value variation calculation section which calculates a variation of a pixel 
value at each point in pixel value domain of the image based on the embedded position in frequency domain determined 
by the embedded position selection section and on the variation of a frequency component value determined by the 

ss frequency component value variation decision section; a pixel value conversion section which embeds watermark 
information by changing corresponding pixel value data of the image in accordance with the variation of a pixel value 
at each point in pixel value domain calculated by the pixel value variation calculation section; and an output section 
which outputs pixel value data of the image into which watermark information has been embedded. 
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[0046] As described above, with this constitution, watermark infoimation is embedded not in frequency domain as 
in the conventional device but in pixel value domain. To this end, in the above constitution, a pixel value variation 
calculation section is provided to embed watermark information in pixel value domain. The pixel value variation calcu- 
lation section calculates the variation ol a pixel value at each point in pixel value domain, based on the position into 
which watermark information is to be embedded (embedded position) and on the variation of a frequency component 
value at that position. Although the calculation for obtainingthe variation of apixef value requires an arithmetic ope ration 
for obtaining matrix elements of orthogonal transform, the operation quantity is considerably small compared with that 
of orthogonal transform. 

[0047] Here, it is possible to easily calculate the varialion of a pixel value by the arithmetic operation in which when 
it is assumed that a variation of a pixel value at each point in pixel val je domain of the image is Affx), the embedded 
position in frequency domain determined by the embedded position selection section is u and the variation of a fre- 
quency component value determined by the frequency component value variation decision section is AF(u), the pixel 
value variation calculation section calculates an i-th matrix element A-1(xi, ui) of orthogonal transform and calculates 
a variation Af(x) of a pixel value from a calculation result in accordance with a formula: 



Af(x(1) x(n)) 

= A-1(x(1), u(1)).....A-1(x(n}, u(n)) 

20 

-aF(u(1) u(n)), 



where n is an integer equal to or higher lhan 1. 

[0048] In this way, by conducting embedding of watermark information in pixel value domain, the above-stated con- 
25 stitution dispenses with orthogonal transformalion for converting pixel value domain to frequency domain and inverse 
orthogonal transformation for converting frequency domain to pixel value domain conducted after embedding of wa- 
termark information in frequency domain, that is, dispenses with two orthogonal transformations which resuit in great 
complexity -ol computation, thereby making it possible to embed watermark information at high speed. 
[0049] It is noted : however, that the conventional digital watermark technique is capable of determining a watermark 
30 information embedded position and the variation of a frequency component value from the frequency componenl value 
data of an imageobtained by converting pixel value domain tof requency domain, whereas the above-stated constitution 
is not capable of dynamically determining a watermark information embedded position and the variation of a frequency 
component value. Nevertheless, since the above constitution dispenses with the conventionally required twoorthogona! 
transformations and allows the size of a block as a unit for watermark embedding processing to be set sufficiently large 
35 compared with the conventional case, it is possible to diffuse and embed watermark information into a wide range. 
Thus, no problem occurs even if the watermark information embedded position is not determined from the frequency 
component data of the image. 

[OOSO] It is, particularly, more desirable that the device is constituted such that a direct sequence spread spectrum 
method is applied to the above constitution, a pseudo-random number generation section, which generates a pseudo- 

io random number by which a pixel value at each point in pixel value domain of the image is multiplied for spread spectrum, 
is provided, the variation of a pixel value at each point in pixel value domain of the image is calculated by the pixel 
value variation calculation section, based on the embedded position in frequency domain determined by the embedded 
position selection section, on the variation of a frequency component value determined by the frequency component 
value variation decision section and on the pseudo-random number corresponding to each point in the pixel value 

4£ domain of the image and generated by the pseudo-random number generation section. This is because such a con- 
stitution makes it less necessary to dynamically determine the watermark information embedded position and the 
variation of a frequency component value in dependence or the frequency component data of the image. 
[00S1] If this spread spectrum method is employed, the variation of a pixel value can be calculated easily by the 
arithmetic operation in which when it is assumed that a variation of a pixel value at each point in pixel value domain 

so of the image is Affx), the embedded position in frequency domain determined by the embedded position selection 
section is u, the variation of a frequency component value determined by the frequency component value variation 
decision section is AF(u) and the pseudo-random number generated by Ihe pseudo-random number generation means 
is p(x), the pixel value variatbn calculation section calculates an hth matrix element A-1 (xi, ui) of orthogonal transform 
and calculates a variation Af(x) ol a pixel value from a calculation result in accordance with a formula: 



Af(x(1), .... x(n)) 
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•A-1(x(1). u(1)).. 
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... u(n)). 



where n is an integer equal to or higher than 1 . 

[0052] In addition, as for the variation of a frequency component value, it is possible to predetermine a variation 
10 which is higher than the absolute value of frequency component values at almost all embedded positions. In that case, 
no problem occurs even if orthogonal transform is not conducted. 

[00S3] To determine an embedded position, for example, a technique for setting a fixed position in frequency domain 
of the image as an embedded position, a technique for setting a position calculated from user inherent information as 
an embedded position and the like can be employed. Besides, a technique in which a fixed position is not set as an 

IS embedded positron, information on the first number frame is fixed butthe following embedded positions are determined 
based on the information read from the fixed frame (in which technique, watermark information is embedded in the 
first number frame with a predetermined fixed embedded position set as an embedded position, and a certain deter- 
mined calculation is made according to the watermark information in the first number frame, thereby making it possible 
to calculate embedded positions in thefollowingframesj.atechniqueforsettingapositton calculated from user inherent 

20 information as an embedded position, a technique for passing an embedded posilion by means other than watermark 
information (in which technique, the method of calculating a watermark information embedded position is preset, the 
data used for embedding of watermark information and necessary for the calculation is fed to the digital watermark 
detection device not as watermark information embedded into image data but as data different from the image data 
andthedigital watermark detection device calc ulates an embedded position from the received data and then determines 

26 the embedded position), a technique for generating an embedded position using random numbers, a technique for 
making the random number series different according to users and the tike can be employed. 
[0054] Further, if a plurality of embedded positions in frequency domain are determined for each block, a beat might 
possibly occur, depending on the embedded positions. Therefore, the embedded position selection section may be 
provided with a function of judging whether or not a beat occurs between a plurality of frequencies (a beat estimation 

30 section). Thus, every time an embedded position candbale is generated, this judging function judges the validity of 
the candidate. Namely, if it judges that the candidate is inappropriate, the candidate is discarded. II it judges that the 
candidate is appropriate, the candidate is selected as an element of a group of embedded positions. To do so, the 
embedded position selection section may consist of a frequency generation section which generates a plurality of 
embedded position candidates and a beat judging section which inputs a plurality of embedded position candidates 

as generated by the frequency generation section, judges the degree of a beat based on the relationships of frequency 
component values among the candidates and selects and ouiputs a group of a predetermined number of embedded 
positions which can avoid the influence of the beat. It goes without saying that the embedded position selection section 
can be originally constituted to determine an embedded position which does not cause a beat (i.e., a watermark fre- 
quency). 

40 .[0055] Additionally, as for the variation of a frequency component value, a variation which is preset based on the 
frequency distributions of frequency component values for a plurality of images may be used. It is more desirable to 
employ a constitution that a digital watermark detection section, which calculates the frequency component value at 
the watermark information embedded position based on the image into which the watermark information is embedded 
by the pixel value conversion section and which feeds back the frequency component value or watermark information 

45 recovered from the frequency component value to the frequency component value variation decision section, is pro- 
vided and that the variation of a frequency component value is adaptivaly changed based on the frequency component 
value or the watermark information fed back by this digfta! walermark detection section. It is further more desirable to 
employ a constitution that a filtering section, which inputs the image into which watermark information is embedded 
and which conducts predictable data operation to the image, is provided between the pixel value conversion section 

so and the digital watermark detection section, that the digital watermark detection section calculates a frequency com- 
ponent value at the watermark information embedded position based on the image which has been subjected to this 
data operation, and that the frequency component value or the watermark information recovered from the frequency 
component value is fed back to the frequency component value variation decision section. 

[0056] Furthermore, it is possible to employ a constitution that a frequency component value calculation section, 
5$ which calculatesafrequency component value at each embedded posit ion determined by the above embedded position 
selection section based on a pixel value at each point in the pixel value domain of the image, is provided and that the 
frequency component value variation decision section determines the variation of a frequency component value at 
each embedded position based on the frequency component value at each embedded position. If the spread spectrum 
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method is used, a frequency component value calculation section, which calculates a frequency component value at 
each embedded position determined by the embedded position selection sectbn based on each embedded position, 
on a pixel value at each point in the pixel value domain of the image and on ihe pseudo-random number generated 
by the pseudo-random number generation section and corresponding to each point in the pixel value domain of the 
s image, may be provided and ihe frequency component value variation decision section may determine the variation 
of a frequency component value at each embedded position based on the frequency component value at each em- 
bedded position. 

[0057] In this way, by obtaining a frequency component value at an embedded position, i.e., frequency component 
values at a small number of points and thereby determining the variation of a frequency component value, it is possible 
J" to make a setting such that perceived influence on the image may be reduced and that the variation of a frequency 
component value due to embedding may be suppressed to be small while using information on general properties 
about the variation of a frequency component value under various operations (such as a property that the variation of 
any frequency component value falls within a certain range). 

[0053] Next, a digital watermark detection device for inputting pixel value data of an image into which watermark 
information is embedded and detecting the watermark information, according to the present invention is characterized 
by comprising an embedded position selection section which determines a position in frequency domain into which 
the watermark information is embedded; a frequency component value calculation section which calculates afrequency 
component value at each embedded position based on pixel i/alue data of an image into which the watermark infor- 
mation has been embedded and on each embedded position determined by the embedded position selection section; 

.20 and a watermark information recovery section which recovers Ihe watermark information based on the frequency com- 
ponent value at each embedded position calculated by the frequency component value calculation section. 
[0059] In this way, with the above constitution, a frequency component value at each embedded position determined 
by the embedded position selection section is calculated based on a pixel value at each point in the pixel value domain 
of the image into which watermark information has been embedded and watermark information is recovered based on 

£5 the frequency component value al the embedded position. Due to this, compared with the conventional detection device 
which converts the pixel value data of the image into which watermark information is embedded from that in pixel value 
domain to that in frequency domain by orthogonal transform, acquires frequency component data forming the image 
data in frequency domain and detects watermark information from the frequency component value data, the device 
according to the present invention does not require cond jcting orthogonal transform to thereby decrease the operation 

so quantity and to allow high-speed processing. 

[0060] Here, the frequency component value can be easily calculated by the arithmetic operation in which when it 
is assumed that a pixel value at each position in pixel value domain of the image into which the watermark information 
is embedded is f(x) and the embedded position in frequency domain determined by the embedded position selection 
section is u, the frequency component value calculation section calculates an i-th matrix elemenl A(ui, xi) of orthogonal 

35 transform and calculates a frequency component value F(u) at the embedded position u from a calculation result in 
accordance with a formula: 

AF(u(1),...,u(n)) 

40 

= £x(0)...2x(n)A-1 (x(1 ), u(1 ))-...-A-1 (x(n), u(n)) • Af{x(1) x(n)). 

[0061] Further, in the digital watermark detection device which employs a direct sequence spread spectrum method 
and treats an image into which watermark information is embedded as a target for detection of the watermark infor- 
ms mation, a frequency component value at each embedded position is calculated by the frequency component value 
calculation section based on the pixel value data of the image into which the watermark information is embedded, on 
each embedded position determined by the embedded position selection section and on the pseudo-random number 
series used for spread spectrum. Thus, high-speed processing can be realized. 

[0062] Here, the frequency component value can be easily calculated by the arithmetic operation in which when it 
so is assumed that a pixel value at each position in pixel value domain of the image into which the watermark information 
is embedded isf (x), the embedded position in frequency domain determined bythe embedded position selection section 
is u and the pseudo-random number generated by the pseudo-random number generation means is p(xi), thef requency 
component value calculation section calculates an i-th matrix element A(ui, xi) of orthogonal transform and calculates 
a frequency component value F(u) at the embedded position u from a calculation result In accordance with a formula: 

55 

AF(u(1), .... u{n)) 
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= Sc(0)_.ae(n) p(x(1), .... x{n)) 
•A-1(x(1), u(1)).....A-1(x{n), u(n)) 
■AHx(1) x(n)). 

[0063] Meanwhile, it is possible to constitute a digital wateitnark embedding and detection device provided with both 
the digital watermark embedding function of the above digital watermark embedding device and the digital watermark 
detection function of the above digital watermark detscton device. In this device, parts common to the both lunctions 
are shared between them, thereby making the constitution simple. 

[00G4] Moreover, the present invention is characterized by comprising an input section which inputs pixel value data 
of an image which is a target for embedding watermark information; a frequency component pair generation section 
which determines a pair of frequency components at positions in frequency domain into which the watermark informa- 
tion is to be embedded, irrespectively of a frequency component value; a pseudo-random number generation section 
which generates a pseudo-random number by which a pixel value at each point in pixel value domain of the image is 
multiplied for spread spectrum; a direct sequence spectrum spreading section which outputs pixel value data spectrum- 
spread by multiplying pixel value data of the imags by the pseudo-random number generated by the pseudo-random 
number generation section; an orthogonal transformation section which orthogonally transforms the pixel value data 
spectrum-spread by the direct sequence spectrum spreading section and generates frequency component value data; 
a first component change section which changes a frequency component value, generated by the orthogonal trans- 
formation section, of the first component in the pair of frequency components at the embedded positions generated 
by the frequency component pair generation section, using a valued the watermark information; a second component 
change section which changes a frequency component value, generated by the orthogonal transformation section, of 
the second component in the pair of frequency components at the embedded positions generated by the frequency 
component pair generation section, using the value of the watermark information, the second component change sec- 
tion changing the frequency component in an opposite direction to that for the change by the first component change 
means; a reverse-orthogonal transformation section which reverse-orthogonally transforms the frequency component 
value data changed by the first component change section and the second component change section, and which 
outputs pixel value data; an inverse spectrum spreading section which conducts inverse spectrum spreading by mul- 
tiplying the pixel value data outputted from the reverse-orthogonal transformation section by the pseudo-random 
number generated by the pseudo-random number generation section, and which outputs pixel value data of the image 
into which the watermark information has been embedded; and an output section which outputs the pixel value data 
of the image into which the watermark information has been embedded. 

[0065] As described above, Ihe constitution of the device is characterized in thai the patchwork method is not directly 
applied to the pixel value data of an image but that direct sequence speclrum spreading is conducted by multiplying 
the pixel value data of the image by the PIM sequence random number value, the resultant image data is subjected to 
orthogonal transformation, a pair of frequency components are generated at random for the transformed image data 
and thereafter the patchwork method is applied. 

[0066] The frequency component data obtained by subjecting the image data to orthogonal transformation by direct 
sequence spectrum spreading has statistically almost uniform distribution close to normal distribution. Therefore, even 
if the number of frequency component pairs at embedded positions selected at random is small, the statistically sig- 
nificant difference occurs between the frequency component value distributions of the first component (one component) 
and the second component (the other component) of the pairs after change operations of the first and second compo- 
nent change sections, by the change operations, thereby making significant watermark embedding possible. This fa- 
cilitates estimating, in advance, the necessary number of pixel pairs to be generated during embedding of watermark 
information. 

[0067] Furthermore, a digital watermark detection device, which employs a new patchwork method stated above, 
inputs the pixel value data of an image into which watermark information is embedded and detects watermark infor- 
mation, a digital watermark detection device for inputting pixe; value data of an image into which watermark information 
is embedded and detecting the watermark information, accordingto 1he present invention is characterized by compris- 
ing a frequency component pair generation section which determines a pair of frequency components at positions in 
frequency domain into which the watermark information is to be embedded, irrespectively of a frequency component 
value; a pseudo-random number generation section which' generates a pseudo-random number by which a pixel value 
at each point in pixel value domain of the image is multiplied for spread spectrum; a direct sequence spectrum spreading 
section which outputs pixel value data spectrum-spread by multiplying pixel value data of the image at each point in 
pixel value domain by the pseudo-random number generated by the pseudo-random number generation section; an 
orthogonal transformation section which orthogonally transforms the pixel value data spsctrum-spread by ihe direct 
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sequGnce spectrum spreading section and generates frequency component value data; a first component average 
calculation section which calculates an average of frequency component values, generated by the orthogonal trans- 
formation section, of the first component in the pairs of frequency components at respective embedded positions gen- 
erated by the frequency component pair generation section; a second component average calculation section which 

5 calculates an average of frequency component values, generated by the orthogonal transformation section, of the 
second component in the pairs of Irequency components at respective embedded positions generated by the frequency 
component pair generation section; and an average comparison section which compares the two averages calculated 
by the first component average calculation section and the second component average calculation section, and recov- 
ers the watermark information based on the comparison result. 

10 [0063] As stated above, if conducting direct sequence spectrum spreading by multiplying the pixel value data of the 
image into which watermark information is embedded by the PN sequence random number value, subjecting the re- 
sultant image data to orthogonal transformation and Dale ulating averages ol the first component and second component 
in the frequency component pair at the watermark embedded positions, respectively, then averages close to normal 
distributions cart be obtained. This allows the embedded watermark information to be recovered (detected) with high 

is accuracy by comparing the averages and facilitates estimating, in advance, the necessary number of'pixel pairs to be 
generated during the detection of watermark information. 

[0069] As described so far in detail, according to the present invention, during embedding of watermark information, 
even if a large blocksize is selected, calculation costs reqjired for direct sequence spectrum spreading, inverse spec- 
trum spreading, orthogonal transform and reverse-orthogonal transform can be made lower, therefore, high-speed 
20 embedding of watermark information can be realized. Owing to this, while there has been conventionally no other 
choice but to mount the device as hardware in view of performance, it is possible to mount the device or functions as 
software according to the present invention. In addition, if the function of embedding watermark information is to be 
realized by a hardware circuit, it can be realized at lower hardware cost than the conventional cost. 
[0070] Further, according to the present invention, it is possible to prevent the loss of watermark information against 
various operations including, for example, D/A conversion and A/D conversion while a large block size is selected and 
the visual degradation of the image is suppressed to be small. 

[0071] Moreover, according to the present invention, by embedding watermark information at high speed, it is possible 
to realize a fingerprinting device for embedding information on users and transfer destinations when data such as an 
image is used and transferred, 

so [0072] Additionally, accordingto the present invention, by detecting watermark inlormation al high speed, it is possible 
to apply the present invention to a case of embedding information on control of use and copy control and descrambls 
keys into data such as image, totransfer these pieces ol information together with the data and to detect the information 
inslantly when they are used. 

[0073] Accordingto the present invention, by detecting and embedding a digital watermark at high speed, use method 
35 such that the use control, copy control information, and descramble key are extracted from the data of the image etc., 
the watermark information is erased immediately, another watermark information is superimposed, and watermark 
information is altered in another watermark information based on those information is possible, when distributing and 

using. 

[0074] Furthermore, according to the present invention, even if the orthogonal transformation in which FFT is used 

40 is not performed, since a high-speed embedding or detectbn is possible, the size ol the block need not be set in the 
power of two, and the block of an arbitrary size can be selected. Therefore, to the image and the block with a size 
different from the power of two, special processing need not be perlormed about the remainder or the lack part. In 
addition, when watermark information of arbitrary size (n) is embedded into the image oi arbitrary size [W X H) by 
using this character, special processing 1o the remainder and the lack part caused by division need not be performed 

AS by the block division of w = W / n1 , h - H / n2 and n1 Xn2Sn. 

[0075] Furthermore, according to the present invention, the patchwork method is not directly applied to the pixel 
value data of an image. But, after spread spectrum is conducted by multiplying the pixel value data of the image by 
the PN sequence random number, the resultant image data is subjected to orthogonal transformation, a pair of fre- 
quency components at embedded positions are generated at random for the transformed image data and then the 

5° patchwork method is applied. By doing so, even if the number of frequency component pairs at embedded positions 
is small, the statistically significant different occurs between the distributions of frequency component values of the 
first and second components in the pair after change operations are conducted to the first and second components, 
respectively. As a result, it is possible to realize significant watermark embedding and highly accurate Watermark de- 
tection, thereby providing higher-speed processing. 

ss [0076] This summary of the invention does not necessarily describe ali necessary features so that the invention may 
also be a sub-combination of these described features. 

[0077] The invention can be more fully understood from the following detailed description when taken in conjunction 
with the accompanying drawings, in which: 
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FIG. 1 is a block diagram showing the constitution of a conventional digital watermark embedding device for em- 
bedding watermark information in frequency domain; 

FIG. 2 is a block diagram showing the constitution of a conventional digital watermark detection device for detecting 
embedded watermark information in frequency domain by the digital watermark embedding device shown in FIG. 1 ; 
$ FIG. 3 is a block diagram showing the constitution of a conventional digital watermark embedding device for em- 

bedding watermark information into a spread spectrum image in frequency domain; 

FIG. 4 is a blockdiagram showing the constitution of a conventional digital walermark detection device tor detecting 
watermark information embedded in Ihe frequency domain of the spread speclrum image by the digital watermark 
embedding device shown in FIQ. 3; 
10 FIG. 5 is a blockdiagram showing the constitution of a conventional digital watermark embedding device employing 

a patchwork method; 

FIG. 6 shows the distribution of pixel values of all pixels before embedding of watermark information and that of 
pixel values of the paired first and second components of a pixel after embedding by the digital watermark em- 
bedding device shown in FIG. 5; 
15 FfG. 7 is a block diagram showing the constitution of a conventional watermark detection device for detecting 

watermark information embedded by the digital watermark embedding device shown in FIG. 5; 
FIG. 8 is a block diagram showing the constitution of a digital watermark embedding device in a first embodiment 
according to the present invention; 

FIGS. 9A and 9B show an example of the constitution of an embedded position selection section 11 shown in FIG. 
20 8 and a flow chart for describing the operation of the embedded position selection section 11 , respectively: 

FIG. 10 is a view for describing a way to appropriately sel embedding intensity A. using the frequency distribution 
of frequency component values F; 

FIG 11 is a block diagram showing an example of the constitution of a pixel value variation calculation section 1 3 
shown in FIG. 8; 

25 FIG. 1 2 is a block diagram showing the constitution of a digital watermark detection device in the first embodiment 

according to the present invention; 

FIG. 13 is a block diagram showing an exampte of the constitution of a frequency component value calculation 
section 22 shown in FIG. 12; 

FIG. 1 4 is a block diagram showing the constitution of a digital watermark embedding device employing a spread 
30 spectrum technique in a second embodiment according to the present invention; 

FIG. 1 5 is a block diagram showing an example of the constitution of a pixel value variation calculation section 33 
shown in FIG. 14; 

FIG. 16 is a block diagram showing the constitution of a digital watermark detection device employing the spread 
spectrum technique in the second embodiment according to the present invention; 
ss FIG. 17 is a block diagram showing an example of the constitution of a frequency component value calculation 

section 42 shown in FIG. 16; 

FIGS. 18A and 1BB are block diagrams showing the constitution of a digital watermark embedding device for 
embedding adaptive watermark information based on the detection of watermark information in a third embodiment 
according to the present invention; 
40 FIGS. 1 9A and 19B are views for describing the function of a frequency component value variation decision section 

51 2 provided in a digitat watermark embedding section 51 shown in FIGS. 18A and 183 and determining a fre- 
quency component value variation based on feedback information; 

FIGS. 20A and 20B are block diagrams showing an example of the constitution of a digital watermark embedding 
device for embedding watermark information by feeding back the distortion of an image and adaptively changing 
4S a frequency component value variation; 

FIG. 21 is a block diagram showing the constitution of a digital watermark embedding device having an embedding 
intensity decision function based on afrequency component value at an embedded position in a fourth embodiment 
according to the present invention; 

FIG. 22 shows the variation of the frequency component vaiue of an image by compression/expansion operation; 
50 FIG. 23 is a view lor describing how to apply the variation of a frequency component value to decrease perceived 

influence on an image; 

FIG. 24 is a block diagram showing the constitution of a digital watermark embedding / detection device having a 
digital watermark detection function in addition to the same digital watermark embedding function as that of the 
digitai watermark embedding device shown in FIG. 21; 
ss FIG. 25 is a block diagram showing the constitution of a digital watermark embedding device employing an ad- 

vanced patchwork method in a fifth embodiment according to the present invention; 

FIG. 26 is ablock diagram showing the constitution of a digital watermark detection device for detecting watermark 
information embedded by employing the advanced patchwork method in the fifth embodiment according to the 
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present invention; and 

FIG. 27 is a block diagram which shows the outline configuration ol digilal walermark embedding and detection 
device which rewrites copy control information, generates new copy control information based on the copy control 
information, and embeds the new copy control information. 

5 

[0078] Now, the embodiments ol the present invention will be described with reference to the drawings while taking 
the embedding of watermark information into image data and the detection thereof as an example. 

[First Embodiment] 

to 

[0079] FIG. 8 is a block diagram showing the constitution of a digital watermark embedding device in the first em- 
bodiment according to the present invention. 

[0030] In FIG. 8, s digital watermark embedding device (WM embedding device) 10 inputs the pixel value data of an 
image which is atargetfor watermark embedding, embeds watermark information into the image and outputs the pixel 
is value data of the image into which the watermark information has been embedded. In this digital watermark embedding 
device 1 0, the data of an entire image or data in units of frames of the image may be inputted and outputted or the 
data may be continuously inputted and o.utputted as a bit stream. This is similar to a digital watermark embedding 
device 30 in the second embodiment to be described later 

[0081] The digital watermark embedding device 10 consists of an embedded position selection section 11, a fre- 
20 quency component value variation decision section 12, a pixel value variation calculation section 13 and a pixel value 
conversion section 14. 

[0062] The embedded position selection section 11 determines a position in frequency domain (embedded position) 
to embed watermark information without depending on a frecuency component value. The following description will 
be given while assuming that there is one embedded position (u, v). However, it is possible to provide a plurality of 
25 embedded positions, in the latter case, the embedded position selection section 11 provides a group S f of embedded 
positions (u, v) in frequency domain. 

[0063] The embedded position selection section 11 employs a technique in which, for example, a fixed position of 
an image in frequency domain is set as an embedded position to determine (select) a watermark information embedded 
position. It is also employ othertechniques including a technique in which a fixed position is not set as an embedded 

so position but information on the first number frame is fixed but later positions are determined based on information read 
by the fixed frame, lhat in which a position calculated from user inherent information is set as an embedded position, 
that 1or providing an embedded position using means otherthan watermark information, that for forming an embedded 
position by a random number, that for making random number series different according to users and the like 
[0084] Further, a plurality of embedded positions may be determined so as to avoid generating a beat between the 

35 walermark information embedded into a plurality of positions by Hie embedded position selection section 11. For ex- 
ample, if a plurality of embedded positions formed by random numbers have extremes at the same point or a neigh- 
borhood point in frequency domain, it is sometimes possible to visually recognize that watermark information is em- 
bedded. If the overlap of the extremes is not a point but a region spreading one-dimensionally or two-dimensionaliy, 
In particular, the embedding of watermark information may be easily recognized. Then, a function of judging whether 

io or not a distinct beat occurs between a plurality of embedded positions may be provided to the embedded position 
selection Section 11. With this constitution, whenever a candidate for an embedded position is generated, the validity 
of the candidate is judged using this judging function. If it is judged as an invalid candidate, the candidate is removed, 
if it is judged as a valid candidate, it is selected as an element an the group S ( of embedded positions. 
[0085] An example of the constitution of the embedded position selection section 11 is shown in FIG. 9A. In FIG. 9A, 

4£ the embedded position selection seclion 11 consists of a frequency generation section 111 generating a plurality of 
pairs of watermark frequencies determined by, for example, random numbers as candidates for embedded positions 
and a beat judgment section 112 inputting the generated plural embedded position candidates, judging the degree of 
a beat, selecting a group S f of a predetermined number (N) of embedded positions which may avoid the influence of 
the beat and outputting the group 

so [0086] Here, the concrete operation of the embedded position selection section 11 in a case where the embedded 
position selection section 11 is constituted as shown in FIG. 9A will be described with reference to the flow chart of 
FIG. 9B. 

[0087] First, the beat judgment section 11 2 sets a variable i indicating an embedded position at an initial value 0 and 
initializes a buffer (list) storing the group S f of embedded positions (Uj, v$ (in a step S1 ). The position (u ; , v r ) =(0,0) is 
ss excluded from the candidates. 

[0088] In this state, the beat judgment section 112 generates, by tbe frequency generation section 111, one candidate 
for an embedded position consisting of a pair of frequencies determined by random number, inputs the embedded 
position candidate and sets the candidate as (u;, V|) (in a step S2). 
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[0039] Next, the beat judgment section 1 1 2 sets a variable j indicating an embedded position as a comparison to the 
embedded position candidate (Uj, v s ) for estimating and judging thedegree of a beat in respect ol the embedded position 
candidate (Uj, Vj) at an initial value of 0 (in a step S3). This variable j indicates the j-th selected embedded position (the 
j-th embedded position in the group S f ). 
5 [0090] Then, the beat judgment section n 2 compares the variables i and j (in a step S4). If the comparison result 
is not i < j as shown in FIG. 9B, the embedded position candidate (u h v,) of this time is selected as an embedded 
position and sets It as the J-th position in the group S ( (in a step S5). 

[0091] Here, if the number of embedded position candidates set in the S f is N, i.e., i does not reach i = N - 1 (in a 
step SB), the beat judgment section 112 increments i by 1 (in a step 57). Then, the beat judgment section 112 causes 
io the frequency generation section 111 to generate Ihe next embedded position candidate and inputs and sets the em- 
bedded position candidate as (Uj, Vj) (in the step S2). 

[0092] Next, the beat judgment section 112 sets the variable j at the initial value of 0 (in the step S3). If the variable 
j is lower than i, the section 112 judges whelher or not the embedded position candidate (Uj, v,) inputted in the step S2 
of this time has the influence of a beat on the j-th selected embedded position (Uj, Vj) ; i.e., (uj, Vj) (in a slep S8). In olher 
is words, since a wave (watermark) having two or more different frequencies never fails to cause a distinct beat, the beat 
judgment section 112 judges the degree of a beat as follows for Ihe purpose of suppressing a beat generated region 
to a small region or to a small dimension. 

[0093] First, if two or more frequencies in a frequency group have an rational-ratio to one another, a large beat with 
a frequency o) the least common multiple of their frequencies occurs. For example, in an image spreading two-dimen- 
20 sionally, if an rational ratio is established between the two frequencies of watermark, the peak of this beat indicates a 
grid-like region spreading one-dimen sionally. The beat judgment section 112 judges whether or not such an rational 
ratio or an approximate rational ratio is established among a plurality of frequencies and uses the result to judge the 
degree of a beat. 

[0094] Specifically, Ihebeal judgment section 112 judges whelher or not the influence of a beat is present by judging 
£5 whether or not the following relationship between the component (Irequency) of the embedded position candidate (Uj, 
v s ) and that of the j-th selected embedded position (Uj, Vj): 

(u, xv j -v [ xu j )/{l(u j ,v i )lxl(u j ,v j )l} 

SO 

satisfies the following judging condition: 

-e<(u j xv j -v i xu j )/[l(u i ,v s )l 

35 

x Ku,,Vj)l}< + e, 

where e is a positive number satisfying 1 » e and may be set at an appropriate value in advance and l(u, v)l indicates 
the magnitude of a vector (u, v). 

to [0095] Namely, if the two frequency components (Uj, Vj) and (uj, Vj) are vectors in almost the same direction, a beat 
causes a one-dimensional or two-dimensional peak region. Thus, to remove a set of frequency components having 
the above relationship, the beat judgment section 112 makes an judgment according to the above judging condition. 
[0096] Now, if the relationship between the component (frequency) of the embedded posilion candidate (Uj, Vj) and 
thai of the j-th selected embedded position (uj, Vj) satisfies the above judging condition, the beat judgmenl section 112 

as judges that a large beat occurs between the embedded position candidate (Uj, Vj) and the embedded position (uj, Vj) 
and processing returns to the step S2 without selecting the candidate. Then, the beat judgmenl section 112 causes 
the frequency generation section 111 to generate the next embedded position candidate, inputs and sets the embedded 
position candidate as (u f , v ; ) and conducts processings after step S3. 

[0097] if the relationship does not satisfy the above condition, the beat judgment section 112 judges that a large beat 
SO does not occur between the embedded position candidate (u r Vji and the embedded posilion (u ( , v,) and increments 
Ihe variable j by 1 (in a step S9). If the value of j which has been incremented by 1, is equal to or higher than i, the 
beat judgment section 112 judges that it can confirm that the embedded position candidate (u,, Vj) of this time does not 
have the influence of a beat on all of the already seiected embedded positions (i.e. , (i - 1) embedded positions), selects 
ihe embedded position candidate (Uj, Vj) as an embedded position and sets the candidate in the j-th position of S f (in 
£5 the step S5). On the other hand, if the value ol the variable j which has been incremented by 1, is still lower than the 
variable i, the beat judgment section 112 goes to a step S8 of judging whether or not the embedded position candidate 
(Uj, Vj) of this time has the influence of a beat on the j-th selected embedded position (u;, Vj), i.e., (u- t , Vj). 
[0098] Thereafter, if the number of embedded positions set in the S, becomes IM (in the step S6), the beat judgment 
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section 1 1 2 outputs the content ot S f , that is, a group of N embedded positions to the pixel value variation calculation 
section 1 3 (in a step S1 0). 

[0099] In this way, no beat indicating a peak spreading one-dimensional ly or two-dimensionally takes place between 
the respective components of the N embedded positions oulputted from the beat judgment section 112 (in theembed- 
s ded position selection section 11 ). It is noted that the procedures in accordance with the ftaw chart of FIG. 9B are given 
only as an example. In short, N embedded positions which are not likely to cause a perceptually distinct beat between 
the respective components may be selected as embedded positions from embedded position candidates sequentially 
generated by the frequency generation section 111. 

[01 00] The judgment of the presence of the influence of the beat of the above-mentioned flowchart is one example, 
to and a different judgment method can be adopted. Whether the rational-ratio between the frequencies exists is judged 
in the above-mentioned example, bul can be judged from the amplitude of the beat. Specifically, the sum of embedding 
strengths concerning the embedded positions where a rational ratio exist is obtained, and may be judged by comparison 
between the sum and certain predetermined threshold value. 

[01 01] The above is similar to a case where a group of embedded positions are outpulted by an embedded position 

1S selection section 31 in the second embodiment to be described later. 

[0102] The frequency component value variation decision section 12 determines to which values the respective 
embedded positions (u, v) selected by the embedded position selection section 11 should be changed. In this case, a 
variation (embedding intensity X) indicating embedded watermark information is determined (selected). For the selec- 
tion of this variation, a technique in which watermark information is denoled by, for example, binary numbers and a 

so variation indicating a predetermined value of 0 and a variation indicating 1 are selected in accordance with the binary 
denotation, can be employed. The embedding intensity X can be selected without requiring the frequency component 
value of an image, by setting the intensity which is not dependent on the average ol the frequency component values 
of the image and those of multiple points. For example, while setting, as embedding intensity X, a value (a positive 
value) higher than a typical frequency component value, if a variable indicates 1 , the embedding intensity X is selected 

25 as variation and if a variable indicates 0, a vafue obtained by changhg the code of the embedding intensity X can be 
selected as variation. In this case, in the dstection of watermark information, if a frequency component value is a 
positive value, it can be judged that 1 is embedded and if ft is a negative value, it can be judge that 0 is embedded. 
[01 03] FIG. 1 0 is a viewfor describing the appropriate setting of embedding intensity A. using the frequency distribution 
of frequency component value F (u, v). In FIG. 10, a frequency distribution 151 shows a case where watermark em- 

30 bedding by means of direct sequence spectrum spreading (SS) is not applied and a frequency distribution 1 52 shows 
a case where watermark embedding by means of direct sequence spectrum spreading (SS) is applied in the second 
embodiment to be described later. By obtaining frequency distributions of a plurality of typical sample images, it has 
an advantage of being possible to set variation AF(u,v) of the frequency component to become an appropriate value 
for most images. Moreover, for a moving picture, it has an advantage that variation AF(u,v) of the frequency component 

35 can be set according to assumption that the frequency distribution of the frame thereafter can be approximated from 
this frequency-distribution concerning one or more frames selected continuously oral suitable intervals. 
[0104] In this embodiment, X is set to be %> IF(u, v) I for almost all embedded positions (u, v). Since the variation of 
a frequency component value is AF(u, v) = +X, the frequency component value after embedding of watermark infor- 
mation is F(u, v) + AF(u, v) = ± X + F(u, v). If 1 is embedded, the frequency component value becomes X + F(u : v) > 0 

AO or a positive value in high probability. If Ois embedded, it becomes -X+ F(u, v) <0or a negative value in high probability. 
[0105] As for the embedding intensity, the technique for using user inherent information and selecting different in- 
tensities for users can be applied as in the case of the embedded position stated above. 

[0106] It is noted that if there are a plurality of embeddedpositions, the frequency component value variation decision 
section 1 2 may be constituted to provide a set A of embedding intensities X corresponding to the respective points in 
is the set S,. 

[0107] The above is also similar to a frequency component value variation decision section 32 in the second em- 
bodiment to be described later. 

[01 08] Now, if the frequency component value decision section 12determinesthe variation of afrequency component 
at the embedded position (u, v) as AF(u, v), the frequency component value F(u, v) at the position (u, v) becomes F 

so (u, v) + AF(u, v) after embedding of watermark information. 

[0109] The pixel value variation calculation section 13 calculates the variation Af(x, y) of a pixel value at each point 
(x, y) in the pixel value domain of an image which is a target for embedding of watermark information, by inputting the 
embedded position (u, v) in frequency domain selected by the embedded position selection section 11 and the variation 
AF(u, v) o( the frequency component value determined by the frequency component value variation decision section 1 2. 

ss [0110] In this case, the pixel variation calculation section 13 calculates the variation Af(x, y) of a pixel value in ac- 
cordance with the following Formula (1 ): 
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Af (x, y) = A" 1 (x, u) - A" 1 (y, v) ■ AF(u, v) (1 ). 

[0111] It is noted that the calculation formula for a case of embedding watermark information two-dimensbnally is 
5 shown in the above calculation of the variation; however, the calculation should not be limited to the two dimensions 
b ut can be appl ied to a one dimension (e. g. , voice data in this case) and three or more dimensions. In the n-dimensiona! 
case, a formula can be expressed as 

10 Af(x (1> x (n> ) 
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where Af (x) is the variation of a pixel value at each point in pixel value domain, AF(u) is the variation of a frequency 
component value and A,' 1 fx,, u,} is the r-th matrix clement of orthogonal transform having, as elements, the position of 
an image in space domain and an embedded position in frequency domain. Symbolsx and u are positional coordinates 
and corresponds to the dimension of the space domain and frequency domain. For brevity, description will be given 
£0 while a two-dimensional image is used as a target. 

[0112] An example of the constitution of the pixel value variation calculation section 1 3 conducing the above calcu- 
lation is shown in FIG 11 

10113] In the example of FIG. 11, the.pixel value variation calculation section 13 consists of the first matrix element 
calculation section 131, the second matrix element calculation section 132 and a variation calculation section 133. 
[0114] The first matrix element calculation section 131 sequentially inputs embedded positions (u, v) in frequency 
domain selected by the embedded position selection section 11 and calculates the first matrix element A -1 (x, u) of 
orthogonal transform. Likewise, the second matrix element calculation section 132 sequentially inputs embedded po- 
sitions (u, v) in frequency domain and calculates the second matrix element A" 1 (y, v) of orthogonal transform. 
[01 1 5] The variation calculation section 1 33 inputs the calculation results of the first matrix element calculation section 
131 and of the second matrix element calculation section 132 and the variation AF(u, v) (= ± X) determined by the 
frequency component value variation decision section 1 2, makes a calculation in accordance with the above Formula 
(1 ) and thereby obtains the variation Affx, y) of a pixel value at each point (x, y) in pixel value domain corresponding 
to the embedded position (u, v} in frequency domain. The variation Af(x, y) of a pixel value at each point (x, y) in pixel 
value domain is fed to the pixel value variation section 14as the output of the pixel value variation calculation section 13. 
[0116] If a plurality of embedded positions, set as (u,, v t ), in frequency domain are selected by the embedded position 
selection section 11 , the variations of pixel values at the respective positions (x, y) in pixel value domain corresponding 
to the embedded positions (Uj, v ( ) are obtained for the embedded positions [u,, v), respectively. The sum of the variations 
for the respective embedded positions (Uj, v,) at the point (x, y), i.e., XA" 1 (x, Uj) ■ AA" 1 (y, Vj) . AF(Uj, V;) may be set as 
the variation Af(x, y) of a pixel value. 
40 [0117] Meanwhile, II pixel values are discrete values such as, for example, integer values, the little variation of a 
pixel value becomes zero due to an error of round-off to an integer value. Owing to Ihis, the pixel value variation 
calculation section 13 may adopt one of stochastically adjacent hieger values instead of simply running down shifts 
from an integer value so that even dispersed and embedded watermark information may not be lost duetothe influence 
of round-off and watermark information can be reproduced. 

[0118] Referring back to FIG, S, the pixel value conversion section 14 converts image data in accordance with the 
variation Af(x, y) of a pixel value at each point (x, y) in pixel value domain fed from the pixel value variation calculation 
section 1 3. 

[011 9] As described above, in this embodiment, processing for embedding of watermark information (or the deter- 
mination of a watermark information embedded position after orthogonal transform required for the processing) can 
be realized without conducting calculations for orthogonal transform, inverse transform required for the conventional 
technique shown in FIG. 1 and those for direct sequence spectrum spreading and inverse spectrum spreading required 
for the conventional technique shown in FIG. 2. Therefore, if the digital watermark embedding device 10 in this em- 
bodiment is emploved, it is possible to sequentially embed watermark information in a real-time manner without the 
need to prepare a targe (mass-storage) buffer even when input images are given as slream data. 
[0120] The above description concerns a case where the entire frame of an image is set as one block and watermark 
information is embedded into the block. It is also possible to embed blocks obtained by dividing a frame into a plurality 
of blocks. 
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[0121] Furthermore, the above description concerns a case where the block as an embedding target is a two-dimen- 
sional block. However, the present invention should not be limited thereto. For example, it is also possible to expand 
blocks to three-dimensional blocks by setting an embedded region including time direction. 

[0122] By doing so, even if wateimark information cannot be sufficiently dispersed and embedded into small blocks 
s in one frame, it is possible to sufficiently disperse and embed watermark information by using a region covering a 
plurality of frames and thereby obtaining sufficient number of pixel points. It is also possible to embed watermark 
information in the form resisiani to thinning out of frames, 

[0123] If blocks are expanded three-dimensionalty and pixel value data in blocks spreading in time direction is stored 
once and then converted to Irequency data, problems such as the occurrence of time delay due to the processing and 
10 heavy-load processing, may possibly occur. In this embodiment, however, due to the fact that watermark information 
can be sequentially embedded into inputted stream data, it is possible to embed watermark information at high speed 
without the need to prepare a large buffer. 

[0124] At that time, for purposes of preventing block boundaries from being visually recognized, variations of pixel 
values as a result of embedding of watermark information may be faded out around the block regions. Since the wa- 
»£ termark information is dispersed and embedded into the entire blocks, it is possible to accurately detect the embedded 
watermark information even with the peripheral portions of the blocks being faded out. 

[0125] The blocks in a three-dimensional image may have an offset so that pieces of every frame can be arranged 
at different positions. The offset may be randomly set. In addition, as long as appropriate orthogonal transform can be 
definsd, the shape of a block need not be always a rectangular parallelepiped. Besides, even if blocks are two-dimen- 
20 sional ones, there is no need to embed watermark information into the blocks in a state in which the blocks are con- 
strained within a frame and watermark information may be embedded freely into the three-dimensional image. 
[0126] FIG. 1 2 is a block diagram showing the constitution of a digital watermark detection device in the first embod- 
iment according to the present inventbn. 

[0127] In FIG. 1 2, a digital watermark detection device {WM detection device) 20 inputs pixel value data of an image 
& which is a target for the detection of watermark information and into which watermark information has been embedded 
by the digital watermark embedding device 1 0 shown in FIG. 8, and detects and outputs watermark information there- 
from. 

[01 28] The digital watermark detection device 20 consists of an embedded position selection section 21 , a frequency 
component value calculation section 22 and an embedded information recovery section 23. 
so [0129] The embedded position selection section 21 has the same function as that of the embedded position selection 
section 11 in the digital watermark embedding device 10 shown in FIG. 8. The section 21 selects the same positron 
(u, v) as the embedded position (u, v) in frequency domain selected by the embedded position selection section 11 
during embedding. Information on the selected embedded position (u, v) is fed to the frequency component value 
calculation section 22. 

as [0130] The frequency component value calculation section 22 calculates a frequency component value F (u, v) at 
the position [u, v) in frequency domain selected by the embedded position selection section 21 according to Ihe following 
Formula (2): 



40 FOl, V) = I x 2 y A(U. X) • A(V. y) • f(X, y) (2) 

Where A(u, x) and A(v, y) are matrix elements of orthogonal transform. 

[0131] An example of the constitution of the frequency componentvaluecalculation section 22 will be shown in FIG. 
13. 

45 [01 32] In the example of FIG. 1 3, the frequency component value calculation section 22 consists of the first matrix 
element calculation section 221, the second matrix element calculation section 222, a frequency component value 
element calculation section 223, an accumulation added value memory section 224 and an addition section 225. 
■ [0133] The first matrix element calculation section 221 sequentially inputs embedded positions (u, v) in frequency 
domain selected by the embedded position selection section 21 and calculates the first matrix element A(u, x) of or- 

50 thogonal transform. Likewise, the second matrix element calculation section 222 sequentially inputs embedded posi- 
tions (u, v) in frequency domain and calculates the second matrix clement A (v, y) of orthogonal transform. 
[0134] The frequency component value element calculation section 223 inputs the calculation results of the first 
matrix element calculation section 221 and of the second matrix element calculation section 222, sequentially inputs 
pixel values f(x, y) at each point (x, y) in pixel value domain corresponding to the embedded positions (u, v) in frequency 

55 domain, and every time inputting them, the section 223 calculates A(u, x) • A(v. y) ■ f(x, y), that is, the element of a 
frequency component value. 

[0135] The accumulation added value memory sectbn 224 is used to store Ihe sum (accumulation value) of A(u. x) 
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■ A(v, y) • f(x, y) calculated by Ihe frequency component value element calculation sectbn 223 to data 
[0136] The addition seclion 225 adds the calculation result of the frequency component value element calculation 
section 223 and the content of the accumulation added vaUe memory section 224 at that time and obtains the sum 
(accumulation value) of A(u, x) • A(v, y) • f(x, y) cabulaled by the frequency component value element calculation 
s section 223 every time the frequency component value element calculation section 223 calculates A(u, x) A(v, y) ■ t(x, 
y). The content of the accumulation added value memory sectbn 224 is updated to this calculation result of the addition 
section 225. 

[0137] II the pixel values f(x, y) of all of the pixels in blocks as an embedding target are inputted, A(u, x) ■ A(v, y) • f 
(x, y) with respect to the final value f(x, y) is calculated by the frequency component value element calculation section 
io £23 and the calculation result of the section 223 and the content of the accumulation added value memory section 224 
{initial value: 0) are added together by the addition section 225, then the addition result Ey Afu, x) • A(v, y) • f(x, y) 
is outputted from the frequency component value calculation section 22 to the watermark information recovery section 
23 as the frequency component value F(u, v) of the embedded value (u, v) selected by the embedded position selection 
section 21. 

is [0136] The watermark information recovery section 23 recovers the embedded watermark information by F(u, v) 
outputted from the frequency component value calculation section 22. This recovery can be realized by estimating that 
1 is embedded if F(u, v) > 0 and 0 is embedded if F(u, v) < 0. 

[Second Embodiment] 

20 

[01 39] FiG. 1 4 is a block diagram showing the constitution of a digital watermark embedding device in the sscond 
embodiment according to the present invention. 

[0140] In FIG. 14, a digital watermark embedding device (WM embedding device) 30 inputs the pixel value data of 
an image which a target for embedding of watermark information (WM information), embeds watermark information 
25 and then outputs the pixel value data on the image after embedding of the watermark information. The digital watermark 
embedding device 3D is characterized by adopting a constitution for embedding watermark information using PN se- 
quence direct sequence spectrum spreading unlike the digital watermark embedding device 10 shown in FIG. 8 in the 
first embodiment. 

[0141] The digital watermark embedding device 30 consists of an embedded position selection section 31 , a fre- 
30 quency component value variation decision section 32, a pixel value variation calculation section 33, a pixel value 
conversion section 34 and a PN sequence generation section 35. 

[0142] The embedded position selection section 31 is the same as the embedded position selection section 11 in 
FIG 8 and determines a position (embedded position) in frequency domain into which watermark information is to be 
embedded without relaying on a frequency component value. Now, description will be given while it is assumed that 
there is one embedded position (u, v). 

[0143] The frequency component value variation selection section 32 is the same as the frequency component value 
decision selection section 12 in FIG 8 and determines to which value a frequency component value at each embedded 
position (u, v) selected by the embedded position selection section 11 is changed. In this case, a variation value (em- 
bedding intensity X) indicating watermark information to be embedded is determined (selected). As described in the 
first embodiment with reference to FIG. 10, X is set to be X > IF(u, V)l for almost all embedded positions (u, v). Since 
the variation AF(u, v) of a frequency component value is AF(u, v) = ± X, the frequency component value after embedding 
of the watermark information becomes F(u, v) + AF(u, v) = ± X + F{u, v). If 1 is embedded, the frequency component 
value is X + F(u, v) > 0 or a positive value. If 0 is embedded, it is -X + F(u, v) < 0 or a negative value. 
[0144] If the frequency component value variation decisbn section 32 determines the variation ol the frequency 
45 component value at the embedded position (u, v) as AF(u, v), the frequency component value F(u, v) at the position 
(u, v) becomes F(u, v) + AF(u, v) after embedding of the watermark information. 

[0145] The PN sequence generation section 35 generates random numbers by which pixel values are multiplied for 
direct sequence spectrum spreading. That is, the PN sequence generation section 35 generates a random value P(x, 
y) for each pixel (x, y) of an image which is a target for embedding of watermark information. If information of a plurality 
so of bits is to be embedded into one block, both a method of embedding all watermark information into image data 
spectrum-spread by the same PN sequence and a method of embedding part of watermark information into data 
spectrum-spread by a plurality of different PN sequence, respectively may be applied. 

[0146] The pixel value variation calculation section 33 calculates the variation A(f (x, y) of a pixel value at each point 
(x, y) in the pixel value domain of an image which is a targel for embedding of watermark information, by inputting the 
ss embedded posilion (u, v) in frequency domain selected by the embedded position selection section 31 , the variation 
AF(u, v) of the frequency component value selected by the frequency component value variation selection section 32 
and the random value p(x, y) of PN sequence generated by the PN sequence generation section 35. 
[01 47] In this case, the pixel value variation calculatton sectbn 33 calculates the variation Af (x, y) of a pixel value in 
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accordance with the following Formula (3): 

Af{x, y) = p(x, y) - A" 1 (x, u) - A _1 (y, u) 

■ AF(U. V) (3). 

[0146] An example of the constitution of the pixel value variation selection section 33 conducting this calculation is 
shown in FIG. 15. 

10 [0149] In the example of FIG. 1 5, the pixel value variation calculation section 33 consists of the first matrix element 
calculation section 331, the second matrix element calculation section 332 and a variation calculation section 333. 
[0150] The first matrix element calculation section 331, which is the same as the first matrix element calculation 
section 131 shown in FIG. 11 in the first embodiment, sequentially inputs Ihe embedded positions (u, v) in frequency 
domain selected by the embedded position selection section 31 and calculates the first matrix element A" 1 (x, u) of 

is orthogonal transform. The second matrix element calculation section 332, which is the same as the second matrix 
element calculation section 132 shown in FIG. 11, sequentially inputs the embedded positions (u, v) in frequency 
domain and calculates the second matrix element A _1 (y, v) of orthogonal transform. 

[0151] The variation calculation section 333 inputs the calculation results of the first matrix element calculation section 
331 and of the second matrix element calculation section 332, the variation AF(u, v) of a frequency component value 

20 determined by the frequency component value variation decision section 32 and the random value P(x, y) of PN se- 
quence generated by the PN sequence generation section 35, calculates the variation Af(x, y) of a pixel value at each 
point (x, y) in pixel value domain corresponding to an embedded position (u, v) in frequency domain in accordance 
with the above Formula (3). The variation Af(x, y) of a pixel value at each point (x, y) in pixel value domain obtained 
by this variation calculation section 333 is fed to the pixel value conversion section 34 as the output of the pixel value 

£5 variation calculation section 33. 

[0152] Referring back to FIG. 14, the pixel value conversion section 34 converts image data (change a pixel value) 
in accordance with the variation Af(x, y) of a pixel value at each point [x, y) in pixel value domain fed from Ihe pixel 
value variation calculation section 33. 

[0153] As described so far, in this embodiment, processing for embedding watermark information (or the determina- 
30 tion of the watermark embedded position and that of the variation of a frequency component value after conducting 
orthogonal transform necessary for the processing) can be realized without conducting calculations for the orthogonal 
transform and inverse transform required forthe conventional technique shown in FIG. 1 and those for direct sequence 
spectrum spreading and inverse spectrum spreading required for the conventional technique shown in FIG- 2. There- 
fore, if the digital watermark embedding device 30 in this embodiment is employed, it is possible to sequentially embed 
35 watermark information in a real-time manner without the need to prepare a large (mass-storage) buffer even if input 
images are applied in the form of stream data. 

[01 54] FIG. 1 6 is a block diagram showing the constitution of a digital watermark information detection section in the 
second embodiment according to the present invention. 

[015S] In FIG. 16, the digital watermark detection device (WM detection device) 40 inputs pixel value data of an 
40 image which is a target for embedding of watermark information and into which watermark information has been em- 
bedded by thedigital watermark embedding device 30 shown in FIG. 14, and detects and outputs watermark information 
therefrom. 

[0156] The digital watermark detection device 40 consists of an embedded position selection section 41 , a frequency 
component value calculation section 42, a watermark information recovery section 43 and a PN sequence generation 
4S section 44. 

[0157] The embedded position selection section 41 , which has the same function as the embedded position selection 
section 31 in the digital watermark embedding device 30 shown in FIG. 14, selects the same position (u, v) as the 
embedded position (u, v) in frequency domain selected while watermark information is being embedded by the em- 
bedded position selection section 31. Information on the selectee embedded position (u, v) is fed to the frequency 

SO component value calculation section 42. 

[01 58] The PN sequence generation section 44, which has the same function as that of the PN sequence generation 
section 35 in the digital watermark embedding device 30 shewn in FIG. 14, generates the same PN sequence as that 
generated by the PN sequence generation section 35 during embedding of watermark information. 
[0159] The frequency component value calculation section 42 calculates the frequency component value F(u, v) at 

ss the position (u, v) in frequency domain selected by the embedded position selection section 41 in accordance with the 
following Formula (4): 
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F(u,v) = £ 5( L y p(x.y)-A(u,x) 

■A{v,y)-f(x,y) (4). 

5 

Where A(u, x) and A(v, y) are the matrix elements of orthogonal transform. 

[0160] An example of the constitution of the frequency component value calculation section 42 is shown in FIG. 17. 
[0161] In the example of FIG. 17, Ihe frequency component value calculation section 42 consists of the first matrix 
element calculation section 421, the second matrix element calculation section 422, a frequency component value 
to element calculation section 423, an accumulation added value memory section 424 and an addition section 425 (cor- 
responding to the first matrix element calculation section 221 , the second matrix element calculation section 222, the 
frequency component value element calculation section 223, the accumulation added value memory section 224 and 
the addition section 225 included in the frequency component value calculation section 22 in the first embodiment, 
respectively). 

15 [0162] The first matrix element calculation section 421 sequentially inputs embedded positions (u, v) in frequency 
domain selected by the embedded position selection section 41 and calculates the first matrix element A(u, x) of or- 
thogonal transform. Likewise, the second matrix element calculation section 422 sequentially inputs embedded posi- 
tions (u. v) in frequency domain and calculates the second matrix element A(v, y) of orthogonal transform. 
[0163] The frequency component value element calculation section 423 inputs the calculation results of the first 

so matrix element calculation section 421 and of the second matrix element calculation section 422, and, at the same 
time, sequentially inputs a pixel value f{x, y) at each point (x, y) in pixel value domain corresponding to Ihe embedded 
position (x, y) in frequency domain and the random value p(x, y) of PN sequence generated by the PN sequence 
generatbn section 44. Every time the section 423 inputs those values, it calculates p(x, y) ■ A(u, x) • A(v, y) • f(x, y), i. 
e., the element of a frequency component value. 

[0164] The accumulation added value memory section 424 is used to store the sum (accumulation value) of p(x, y) 
• A(u, x) ■ A(v, y) • f(x, y) - f{x, v) calculated by the frequency component value element calculation section 423 to date. 
[01 65] Every time p(x, y) • A(u, x) • A(v, y) -f(x, y) is calculated by the frequency component value element calculation 
section 423, the added section 425 adds the calculation, result of the seclion 423 and the content of the accumulation 
added value memory section 424 at that moment and obtains the sum (accumulation value) of p(x, y) ■ A(u, x) ■ A(v, 

30 y) • f ( x . y) calculated by the frequency component value calculation section 423 until that moment. 

[D166] In this way, if the pixel values f(x, y) of ail pixels in the embedding target block have been inputted, p(x, y) ■ 
A(u, x) • A(v, y) . f(x, y) for the final pixel value f(x, y) has been calculated by the frequency component value element 
calculation section 423 and the calculation result of the section 423 and the content of the accumulation added value 
memory section 424 have been added by the added section 425, 1hen the addition result!^ z y p(x, y) • A(u, x) ■ A(v, 

35 y) . f( X> y) is outpuUed, as a frequency component value F(u, v) at the embedded position (u, v) selected by the em- 
bedded posilion selection section 41, from Ihe frequency component value calculation seclion 42 to the watermark 
information recovery section 43. 

[0167] The watermark information recovery section 43 recovers the embedded watermark information by the fre- 
quency component value F(u, v) outputted from the frequency component value calculation section 42. This recovery 
40 can be realized by estimating that 1 is embedded as for, for example, F(u, v) > 0 and that 0 is embedded as for F(u, v) < 0. 

[Third Embodiment] 

[01 68] FIGS. 1 8A and 1 8B are block diagrams showing the constitution of a digital watermark embedding device in 

•45 the third embodiment according to the present invention. 

[0169] A digital watermark embedding device (WM embedding device) 50 shown in FIG. 18A is characterized by 
comprising a digital watermark embedding section (WM embedding section) 51 corresponding to the digital watermark 
embedding device 1 0 in the first embodiment or the digital watermark embedding device 30 in the second embodiment 
and a digital watermark detection section (WM detection section) 52 corresponding to the digital watermark detection 

so. seclion 20 in the first embodiment or the digital watermark detection section 40 in the second embodiment, to adaplively 
determine a parameter such as embedding intensity necessary for watermark embedding processing and thereby to 
realize more assured detection of watermark information and embedding of watermark information having the less 
perceived influence. 

[0170] If watermark information is embedded by the digital watermark embedding section 51 , the digital watermark 
55 detection section 52 instantly detects corresponding watermark information from the image into which the watermark 
information has been embedded. As can be easily understood from ihe constitution of Ihe digital watermark deleclion 
device 20 in the first embodiment (see FIG. 12) or that of the digital watermark detection device 40 in the second 
embodiment (see FIG. 1 6), Ihe frequency component value F(u, v) at the watermark embedded position (u, v) is cai- 
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ciliated (by the frequency component value calculation section 22 in FIG. 12 or the frequency component value cal- 
culation section 42 in FIG. 16). 

[0171] Then, as shown in, for example, FIG. 19A, the frequency component value Is fed back to a frequency com- 
ponent value variation decision section 512 in the digital watermark embedding section 51 (corresponding to the tre- 

5 quency component value variation decision section 12 in FIG. 8 or the frequency component value variation decision 
section 32 in FIG. 14). 

[0172] The frequency component value variation decision section 512 adaptively changes the variation value of a 
frequency component value at an embedded position (u v), that is, watermark information embedding intensity based 
on the frequency component value fed back from the digital watermark detection section 52. 

io [0173] As information fed back to the frequency component value variation decision section 51 2 in the digital water- 
mark embedding section 51 from the digital watermark detection section 52, not a frequency component value but 
watermark information recovered based on the frequency component value may be fed back, in that case, the frequency 
component value variation decision section 51 2 may adaptively change the variation vaiue of a frequency component 
value at an embedded position (u, v), that is, watermark information embedding intensity, based on the watermark 

75 information fed back by the digital walermark detection section 52. 

[0174] In this way, it is possible to increase the probability of accurate detection of newly embedded watermark 
information by causing the digital watermark defection section 52 to instantly deled corresponding watermark infor- 
mation Irom the image into which the watermark information has been embedded by the digital watermark embedding 
section 51 , to feed back the detected watermark information itself or the frequency component value used for the 

20 detection (recovery) o! the watermark information to the digital watermark embedding section 51 and by adaptively 
changing the variation (embedding intensity) of a frequency component value lor later watermark information embed- 
ding processing. 

[01 75] If operation (e.g., image compression/expansion) aimed at the image inlo which watermark information has 
been embedded by the digital watermark embedding section 51 is expected in advance, afiitering section 53 conducting 
25 filtering operation (e.g., compression/ expansion) with the operation being modeled, may be provided as shown in the 
digital watermark embedding device (WM embedding device) 500 shown in, for example, FIG. 1 SB, the digital water- 
mark, detection section 52 may conducts detection in response to the filtering result of the filtering section 53 and the 
detection result may be fed back to Ihe digital watermark embedding section 51. 

[01 76] In addition, the result of evaluatingthe visual (perceived) influence on the image into which watermark infor- 
30 mation has been embedded other than the watermark information detection result can be adapted as information fed 

back to the digital watermark detection section 51 . An example of such a constitution is shown in FIGS. 20A and 20B. 

[0177] In the constitution of FIG. 20A, a perceived information detection section 542 inputting the pixel value of the 

image into which the watermark information outputtsd from the digital watermark embedding section 51 has been 

embedded, and detecting and evaluating the perceived influence (visual distortion compared with the original image 
3s before embedding of watermark Information) on the image, is provided. The evaluation result of the perceived influence 

detection section 541 is fed back to the digital embedding section 51 (e.g., the frequency component value variation 

decision section 512 in the section 51 shown in FIGS. 19Aand 19B). 

[01 78] Meanwhile, in the constitution of FIG. 20B, a perceived influence detection section 542 inputting the variation 
of a frequency component value determined by, for example, the frequency component value variation decision section 

40 512 shown in FIGS. 1 9A and 1 9B in the digital watermark embedding section 51 (i.e., parameter used for watermark 
information embedding in the digital watermark embedding section 51) and detecting and evaluatingthe visual influence 
(distortion) on the image into which watermark influence has been embedded by the digital watermark embedding 
section 51 , is provided. The evaluation result of the perceived influence detection section 542 is fed back to the digital 
watermark embedding section 51 {or the frequency component value variation decision section 51 2 in the section 51 

4S shown in, for example, FIGS. 19A and 19B). 

[0179] An evaluation method using an SM ratio or an evaiuatior method based on a perceived model can be applied 
to the evaluation of perceived influence in the perceived influence detection section 541 or 542. [Fourth Embodiment] 
[0180] In the digital watermark embedding devices in the preceding first and second embodiments, an embedded 
position and embedding intensity are set to be dependent on a frequency component value. In case of the determination 

so of embedding intensify which requires obtaining frequency component values at many positions in frequency domain, 
however, complexity of computation is large and, therefore, it is difficult to select a large block size. Considering this, 
this embodiment adopts a method in which an embedded position does not depend on frequency component values 
and only embedding Intensity depends on those at a few points. 

[0181] FIG. 21 is a block diagram showing the constitution of a digital walermark embedding device in the fourth 

6 embodiment according to the present invention. 

[0182] in FIG. 21, a digital watermark embedding device (WM embedding device) 60 is designed to calculate a 
frequency component value at a watermark information embedded position and determine embedding intensity based 
on the value. The digital watermark embedding device 60 consists of an embedded position selection section 61, a 
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frequency component value variation decision section 52, a pixel value varialion calculation section 63, a pixel value 
conversion section 64, a PN sequence generation section 65, a pixel value read section 56 and a frequency component 
value calculation section 67, 

[0183] The pixel value read section 66 reads a pixel value f(x, y) at each point {x, y) in pixel value domain of an image 

s which is a target for embedding of watermark information. 

[0184] The frequency component value calculation section 67 corresponds to the frequency component value cal- 
culation section 42 in the digital watermark detection device 40 shown in FfG. 16 in the second embodiment. The 
section 67 calculates a frequency component value F(u, v) at the embedded position (u, v) determined by the embedded 
position selection section 61 (corresponding to the embedded position selection section 41 in FIG. 16), based on the 

10 pixel value f(x, y) at each point (x, y) in pixel value domain read by the pixel value read section 66 and the random 
value p(x, y) for the pixel (x, y) generated by the PN sequence generation section 65 (corresponding to the PN sequence 
generation section 44 shown in FIG. 16), in accordance with the Formula (4). If the PN sequence generation section 
65 is no1 provided, that is, if direct sequence spectrum spreading using PN sequence is not used, the frequency com- 
ponent value F(u, v) may be calculated according to the Formula (2). 

is [0185] The frequency component value variation decision section 62 determines (selects) the varistbn AF(u, v) of 
the frequency component value at the embedded position (u, v) based on the frequency component value F(u, v) at 
this embedded position (u, v). 

[0186] The pixel value variation calculation seclion 63 (which corresponds 1o the pixel value variation calculation 
section 33 shown in FIG. 1 4 in the second embodiment) calculates the variation Af(x, y) of a pixel value at each point 
£0 (x, y) in pixel value domain of an image which is a target for embedding of watermark information, by inputting the 
embedded position (u, y) in frequency domain selected by the embedded position selection section 61, the variation 
AF(u, v) of a frequency component value determined by the frequency component value variation decision seclion 62 
and the random value p(x, y) of PN sequence generated by the PN generation seclion 65. 

[0187] The pixel value conversion section 64 converts image data in accordance with the variation Af(x, y) of a pixel 
25 value at each point (x, y) in pixel value domain obtained by the pixel value varialion calculation section 63. 

[0188] The constitution of FIG. 21 is characterized in that the frequency component value variation decision section 
62 can set embedding intensity so that the perceived influence on the image is less and the varialion of a frequency 
component value due to embedding is small while using information on general properties aboul Ihe varialion of a 
frequency component value under various operations. The characteristics will be described with reference to FIGS. 
30 22 and 23. 

[0189] FIG. 22 shows how the frequency component value of an (mage varies by compression and expansion op- 
eration. 

[0190] As can be seen from FIG. 22, the variation of every frequency component value almost falls within a range 
of ± 6. Since this property is seen for almost all image data, the variations of almost ail frequency component values 

35 of any image fall within the range of + 6 as long as a variation 8 having a certain degree of magnitude is set. 

[0191] By using this properly, the frequency component value variation decision section 62 can determine 8-F(u, v) 
in case of embedding 1 and -B-F(u, v) in case of embedding 0 as variations at a time when the digital watermark 
embedding device 60 shown in FIG. 21 embeds watermark information into target data. In this case, the frequency 
component value after embedding, if an error due to round-off during embedding is excluded from the value, is 5 in 

40 case of embedding 1 and -8 in case of embedding 0. The value 5 is normally lower than the embedding intensity \ set 
in the first or second embodiment. While the intensity X roods to be sot higher than a typical frequency component 
value, S can be set as a typical frequency component value, which makes it possible to reduce perceived influence on 
the image. 

[01 92] FIG. 23 shows how to set the variation of a frequency component value to reduce perceived influence on an 
45 image as much as possible 

[0193] In FIG 23, it is assumed that 8' is set slightly higher than 5, the range of a frequency component value is 
divided into partial regions at intervals ol 28', and lhat the i-th partial region, for example, is between (2i - 1)5' and (2i 
+ 1 )5'. 

[0194] In this case, the frequency component value variation decision section 62 selects (2i + 1 )6" - F(u, v) if i is 
so embedded and i is an even number and (2i - 1)8' - F(u, v) if I is embedded and i is an odd number, and selecls (21 - 
1)5" - F(u, v) if 0 is embedded and i is an even number and {2i + 1 )5' - F(u, v) if 0 is embedded and I Is an odd number. 
■ [0195] Now, for the digital watermark detection device to detect corresponding watermark information from the image 
into which watermark information has been embedded by the digital watermark embedding device 60 having a fre- 
quency component value variation decisbn function as slated above, a watermark information recovery section cor- 
ss responding to the watermark information recovery section 43 shown in FIG. 16 may obtain a value of i in (2i + 1) 5' 
which is the proximate frequency component value at the embedded position and judge that 1 is embedded if i is an 
odd number and 0 is embedded if i is an even number. In this method, the absolute value of the variation of a frequency 
componenl value during embedding can be suppressed up to 25' at most. 
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[01 96] The above description concerns a case where the present invention is applied to the digital watermark em- 
bedding device. However, it is also possible to apply the present invention to a digital watermark embedding/detection 
device having both the digital watermark embedding function and the detection function. 

[0197] FIG, 24 is a block diagram showing the constitution of adigital watermark embedding/detection device (WM 
s embedding/detection device) 70 having a digital watermark detection function as well as the same digital watermark 
embedding function as that of the digital watermark embedding device 60 shown in FIG. 21 . !t is noted that the same 
constituent elements are denoted by the same reference symbols as those in FIG. 21 . 

[0198] In FIG. 24, a digital watermark embedding section {WM embedding seciion) 71 corresponding to the digital 
watermark embedding device GO shown in FIG, 21 consists of, as in the case of the digital watermark embedding 
10 device 60, an embedded position selection section 61 , a frequency component value variation decision section 62, a 
pixel value variation calculation section 63, a pixel value conversion section 64, a PN sequence generation section 65, 
a pixel value read section 66 and a frequency component value calculation section 74. 

[01 99] A digital watermark detection section (WM detection section) 72 consists of an embedded position selection 
section 51 , a PN sequence generation section 65, a pixel value read section 36, a frequency component value calcu- 
15 lation section 67 and a watermark information recovery section 73 (corresponding to the watermark information recov- 
ery section 43 shown in FIG. 16). The embedded position selection section 61, the PN sequence generation section 
65, the pixel value read section 66 and the frequency component value calculation section 67 are shared between the 
digital watermark embedding section 71 and the digital watermark detection section 72. 

[0200] As stated above, by employing the constiiution in which part of the functions of the digital watermark detection 
20 section 72 are shared between the section 72 and the digital watermark embedding device 71 , it is possible to prevent 
the digital watermark embedding device 71 and the digital watermark detection device 72 from having overlapped 
functions. This is similar to a constitution without a PN sequence generation section 65. In the latter case, the digital 
watermark embedding device 71 and the digital watermark detection device 72 may share the embedded position 
selection section, the pixel value read section and the frequency component value calculation section therebetween. 
|n addition, if a device which functions as both the digital watermark embedding device 10 shown in FIG. 8 and the 
digital watermark detection device 20 shown in FIG. 20,' is to be reaPzed, or if a device which functions as both the 
digital watermark embedding device 30 shown in FIG. 14 and the digital-watermark detection device 40 shown in FIG. 
16, is to be realized, thentho embedded position selection section may be shared between the sections 10 and 20 
and the embedded position selection section and the PN sequence generation section may be shared between the 
so sections 30 and 40. 

[Fifth Embodiment] 

[0201] FIG 25 is a block diagram showing the constitution of a digital watermark embedding device in the fifth em- 

36 bodiment according to the present invention. 

[0202] In FIG. 25, a digital watermark embedding device (WM embedding device) 180 employs a technique for 
embedding watermark information by means of an advanced patchwork method. Thedevice 180 consists of afrequency 
component pair generation section 1 81 , a PN sequence generation seciion 182, a direct sequence spectrum spreading 
section 183, an orthogonal transformation section 184, the first component change section 185, the second component 

■to change section 186, a reverse-orthogonal transformation section 187 and a direct sequence spectrum spreading sec- 
tion 1 88. 

[0203] The frequency component pair generation section 181 determines (generates) a pair of positions <(au, av}, 
(bu, bv)> in frequency domain into which watermark information should be embedded, irrespectively of a frequency 
component value. 

4B [0204] The PN sequence generation section 182 generates pseudo-random numbers (pseudo-random number se- 
ries) by which a pixel value at each point in pixel value domain of an image which is a target for watermark information 
embedding is multiplied for direct sequence spectrum spreading. 

[0205] The direct sequence spectrum spreading secttan 183 outputs the direct sequence spectrum spreading pixel 
value data by multiplying the image value data of the image by the pseudo-random number generated by the PN 

SO sequence generation section 183. 

[0206] The orthogonal transformation section 1 84 conducts orthogonal transform to the pixel value data spectrum- 
spread by the direct sequence spectrum spreading section 183 and generates Irequency component data. 
[0207] The first component change section 185 changes (increments if watermark information 1 is embedded and 
decrements if 0 is embedded in this case) a frequency component value a, generated by the orthogonal transformation 

ss section 184, in the first component (au, av) in the frequency component pair at each embedded position generated by 
the frequency component pair generation section 181, by, for example, a certain positive value A using a watermark 
information value. 

[0208] On the other hand, the second component changesection 186 changes (decrements if watermark information 
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1 is embedded and increments if 0 is embedded in Ihiscase) a frequency component value b, generated by the or- 
thogonal transformation section 184, in the second component (bu, bv) of the frequency component pair at each em- 
bedded position generated by the frequency component pair generation section 1 81 , by a positive value A in opposite 
direction to 1hat of the first componenl change section 185, using a watermark information value, 
s [0209] The reverse-orthogonat transformation section 187 conducts reverse-orthogonal transformation 1o the fre- 
quency component value data after change by the first component change section and the second component change 
section, and outpuis pixel value data. 

[0210] The direct sequence spectrum spreading section 188 conducts de-spread to multiply the pixel value data 
outputted from the reverse-orthogonal transformation section 167 by Ihe pseudo-random number generated by the 
10 PN sequence generation section 1 82 and outputs pixel vaiue data in;o which watermark information has been embed- 
ded. 

[0211] As described above, in this embodiment, direct sequence spectrum spreading is conducted to the pixel value 
data which is a target for watermark information embedding by the direct sequence spectrum spreading section 183 
and 1hen orthogonal transformation is conducted thereto by the orthogonal transformation section 184. Therefore, the 

is frequency component value data obtained by the orthogonal transformation of the orthogonal transformation section 
1 84 has almost statistically uniform distribution close to normal distribution. Due to this, even if the number of frequency 
component pairs at embedded positions selected at random by the frequency component pair generation section 1 81 
is small, the statistically important difference is seen belween the distributions of the frequency component values of 
the first and second components in the pair after change operation of the first component change section 185 and the 

so second component change section 186, thereby making it possible to carry out significant watermark embedding 

[0212] FIG. 26 is a block diagram showing the constitution of a digital watermark detection device in the fifth embod- 
iment according to the present invention. 

[021 3] In FIG. 26, a digital watermark detection device (WM detection device) 190 inputs the pixel value data of an 
image which is a target for watermark information embedding and into which watermark information is embedded by 
SB the digital watermark embedding device 180 shown in FIG. 25, and detects and outputs watermark information there- 
from. 

[0214] The digital watermark detection device 1 90 consists o1 a frequency component pair generation section 191, 
a PN sequence generation section 192, a direct sequence spectrum spreading section 193, an orthogonal transfor- 
mation section 194, the first component average calculation section 195, 1he second component average calculation 
30 section 1 96 and an average comparison section 197. 

[0215] The frequency component pair generation section 1 91 determines (generates) a pair of positions in frequency 
domain into which watermark information is embedded by the digital watermark embedding device 180 shown in FIG. 
25, irrespectively of a frequency component value. 

[0216] The PN sequence generation section 192 generates a pseudo-random number (or pseudo-random number 
3s series) by which a pixel value at each point in pixel value domain of an image which is a target for Ihe detection of 
watermark information is multiplied for direct sequence spectrum spreading. 

[0217] The direct sequence spectrum spreading section 193 multiplies the image value data of the image by the 
pseudo-random number generated by the PN sequence generation sectbn 192 and thereby outputs direct sequence 
spectrum spreading pixel value data. 

to [021 8] The orthogonal transformation section 1 94 conducts orthogonal transform to the pixel value data spectrum- 
spread by the direct sequence spectrum spreading section 193 and generates frequency component value data. 
[0219] The first component average calculation section 1 95 calculates the average of frequency component values, 
generated by the orthogonal transformation section 194, of the first component in the frequency component pair at 
each position generated by the frequency component pair generation section 191 . The second component average 

45 calculation section 19B calculates the average of frequency component values, generated by the orthogonal transfor- 
mation section 194, of the second componenl in the frequency component pair at each position generated by the 
frequency component pair generation section 191. 

[0220] The average comparison section 197 compares two averages calculated by the firs! component average 
calculation section 1 95 and the second component average calculation section 1 96. The section 1 97 judges that 1 is 

50 embedded if the average of the frequency component values of the fi'st component is higher ihan that ol the frequency 
component values of the second component and that 0 is embedded if the average of the frequency component values 
of the second component is higher, thereby detecting (recovering) the embedded watermark information. 
[0221] The above description concerns a case where the present invention is applied to Ihe embedding of watermark 
information into an image and to the detection of watermark information from the image. The present invention is also 

ss applicable to the embedding and detection of watermark information into and from digital literary properties such as 
sound and music, other than an image. 

[0222] The present invention can be applied to extraction of use control information, in addition, re-embedding (re- 
marking) of digital watermark information based on Ihe informatbn and superimposition of another watermark inlor- 
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rnation to the information elc. 

[0223] An example of applying the present invention to the copy control of DVD (digital video disk) for example will 
be explained. 

[0224] Usually, to prevent an illegal copy, copy control information is embedded to DVD etc. This copy control infor- 
s rnalion is embedded into the disk as digital watermark Information, and, for example, according to embedded informa- 
tion, the control is performed such as copy-enabled, copy-disabled, or copy-enabled for predetermined numbers (e. 
g., once, twice, ...). 

[0225] In this case, to reflect copy conlrol information accurately, il is necessary to read copy control information on 
contents which are the copy target and rewrite copy control information on copied contents appropriately. 

i0 [0226] FIG. 27 is a block diagram which shows the outl in e configuration of digital watermark embedding and detection 
device which rewrites copy control infcrmaiion, generates new copy control information based on the copy control 
information, and embeds the new copy control information. In FIG, 27, since the configuration of digital watermark 
device 10 (or SO etc.} and digital watermark detection device 20 (or 40 e1c.) is similar to the embodiment described 
before, the explanation will be omitted. 

'5 [0227] In FIG. 27, digital watermark in contents read from DVD is read by digital watermark detection device 20. If 
copy control information of read digital watermark information is information that it is possible to copy it only once, 
digital watermark detection device 20 sends ths information to copy controller 80. Copy controllerSO sends copy control 
information rewritten based on copy control information to digital watermark embedding device 10, and digital water- 
mark embedding device 10 embeds this copy control information to contents at the copy destination as digital watermark 

zo information. At this time, digital watermark information may be embedded without erasing digital watermark information 
on the copy origin and without overlapping it, and may be embedded after erasing digital watermark information on 
the copy origin. 

[0228] It is noted that the constitution of the digital watermark embedding device 10 in FIG. 8 and that of the digital 
watermark detection device 20 in FIG. 12 employed in the first embodiment, the constitution of the digital watermark 

£5 embedding device 30 in FIG. 14andthatof the digital watermark detection device40in FIG. 16 employed in the second 
embodiment, the constitutions of the digital watermark embedding devices 50 and 500 in FIGS. 1BA and 18B employed 
in the third embodiment, the constituiion of the digital watermark embedding device 60 in FIG. 21 and that of ihe digital 
watermark embedding/detection device in FIG. 24 employed in the fourth embodiment and the constitution of Ihe digital 
watermark embedding device 180 in FIG. 25 and that of the digital watermark detection device 190 in FIG. 26 employed 

so in the fifth embodiment, can be realized by recording programe for making a computer function as functional means 
which are the constituent element of the computer on a machine-readable recording medium, such as a ROM, a CD- 
ROM, a DVD-ROM, a floppy disk, a hard disk and a memory card, and cau sin g Ihe programs to b e read by th e comp uta r, 
respectively. The programs may be downloaded through a communication line such as a network. 



Claims 

1 . A digital watermark embedding device characterized by comprising: 

40 an input section which inputs pixel value data of an image which is a larget for embedding of watermark 

information; 

an embedded position selection section (11, 31, 61) which determines a position in frequency domain into 
which ths watermark information is to be embedded, irrespectively of a frequency component value; 
a frequency component value variation decision section [12, 32, 202b, 512, 62) which determines a variation 
45 of a frequency component value at each embedded position determined by the embedded position selection 

section; 

a pixel value variation calculation section (13, 33, 63) which calculates a variation of a pixel value at each 
point in pixel value domain of the image based on the embedded position in frequency domain determined by 
the embedded position selection section and on the variation of a frequency component value determined by 
so the frequency component value variation decision section; 

a pixel value conversion section (1 4, 23, 64) which embeds watermark information by changing corresponding 
pixel value data of the image in accordance with the variation of a pixel value at each point in pixel value 
domain calculated by the pixel value variation calculation section; and 

an output seclion which outputs pixel value data of the image into which watermark information has been 
ss embedded. 

2. A digital watermark embedding device characterized by comprising: 
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an input section which inputs pixel value data of an image which is a target for embedding of watermark 
information; 

an embedded position selection section (11, 31, 61) which determines a position in frequency domain into 
which the watermark information is to be embedded, irrespective!/ of a frequency component value; 
a frequency component value variation decision section (12, 32, 202b, 512, 62) which determines a variation 
of a frequency component value at each embedded position determined by the embedded position selection 
seel ion; 

a pseudo-random number generation section (35, 65) which generates a pseudo-random number by which a 
pixel value at each point in pixel value domain ol the image is multiplied for direct sequence spectrum spread- 
ing; 

a pixel value variation calculation section (13, 33, 63) which calculates a variation of a pixel value at each 
point in pixel value domain of the image based on the embedded position in frequency domain determined by 
the embedded position selection section, on the variation of a frequency component value determined by the 
frequency component value variation decision section and on a pseudo-random number corresponding to 
each point in frequency domain of the image generated by the pseudo-random number generation section; 
a pixel value conversion section (1 4, 34, 64) which embeds watermark information by changing corresponding 
pixel value data of the image in accordance with the variation of a pixel value at each point in pixel value 
domain calculated by the pixel value variation calculation section; and 

an output section which outputs pixel value data of the image into which watermark information has been 
embedded. 

A digital watermark embedding device characterized by comprising: 

an input section which inputs pixel value data of an image which is a target for embedding of watermark 
information; 

an embedded position selection section (11, 31, 61) which determines a position in frequency domain into 
which the watermark information is to be embedded, irrespectively of a frequency component value; 
a frequency component value calculation section (22, 42, 67) which calculates a frequency component value 
at the embedded position determined by the embedded position selection section based on a pixel value at 
each point In pixel value domain of the image; 

a frequency component value variation decision section (12, 32, 202b, 512 62) which determines a variation 
of a frequency component value at each embedded posriion determined by ihe embedded position selection 
section based on the frequency component value at the embedded position calculated by the frequency com- 
ponent value calculation section; 

a pixel value variation calculation section (13, 33, 63) which calculates a variation of a pixel value at each 
point in pixel value domain o( the image based on the embedded position in frequency domain determined by 
the embedded position selection section and on the variation of a frequency component value determined by 
the frequency component value variation decision section; 

a pixel value conversion section (14, 34, 54) which embeds watermark information by changing corresponding 
pixel value data of the image in accordance with the variation of a pixel value at each point in pixel value 
domain calculated by the pixel value variation calculation section; and 

an output section which outputs pixel value data of the image into which watermark information has been 
embedded. 

A digital watermark embedding device characterized by comprising: 

an input section which inputs pixel value data of an image which is a target for embedding of watermark 
information, 

an embedded position selection section (11, 31, 61) which determlnes.a position in frequency domain into 
which the watermark information is to be embedded, irrespectively of a frequency component value; 
a pseudo-random number generation section (35, 65) which generates a pseudo-random number by which a 
pixel value at each point in pixel value domain of the image is multiplied for spread spectrum; 
a frequency component value calculation section (22, 42, 67) which calculates a frequency component value 
at each embedded position determined by the embedded position selection section based on the embedded 
position, a pixel value at each point in pixel value domain of the image and a pseudo-random number corre- 
sponding to each point in pixel value domain of the image generated by the pseudo random number generation 
section; 

a frequency component value variation decision section (12, 32, 202b, 512, 62) which determines a variation 
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of a frequency component value at each embedded position determined by the embedded position selection 
section based on the frequency component value cateulated by tie frequency component value calculation 
section, at each embedded position determined by the embedded position selection section; 
a pixel value variation calculation section (13, 33, 63) which calculates a variation cf a pixel value of the image 

5 at each point in pixef value domain based on the embedded position in frequency domain determined by the 

embedded position selection section, on the variation of a frequency component value determined by the 
frequency component value variation decision section and a pseudo-random number generated by the pseu- 
do-random number generation section and corresponding to each point in frequency domain of the image; 
a pixe! value conversion section (14, 34, 64) which embeds watermark information by changing corresponding 

io pixel value data of the image in accordance with the variation of a pixel value at each point in pixel value 

domain calculated by the pixel value variation calculation section; and 

an output section which outputs pixel value data of the image into which watermark information has been 
embedded. 

rs 5. The digital watermark embedding device according to claim 1 or claim 2, characterized in that 

the frequency component value variation decision section sets a variation to be set in advance based on a 
frequency distribution of frequency component values for a plurality of images, 

6. The digital watermark embedding device accord ng to claim i or claim 3, characterized in that 
20 while it is assumed that a variation of a pixel value at each point in pixel value domain of the image is Af (x), 

the embedded position in frequency domain determined by the embedded position selection section is u and the 
variation of a frequency component value determined by the frequency component value variation decision section 
is AF(u), the pixel value variation calculation section calculates an i-th matrix element A -1 (X;, u s ) of orthogonal 
transform and calculates a variation Af(x) of a pixel value from a calculation result in accordance with a formula; 
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Af(X (1) ,.„.X (n) ) 



:A-y i \u (1 V..,AV n \u (n >) 
•AF(u (1) u<">) 



where n is an integer equal to or higher than 1 . 

35 7. The digital watermark embedding device according to claim 2 or claim 4, characterized In that 

while it is assumed that a variation of a pixel value at each point in pixel value domain of the image is Af (x), 
the embedded position in frequency domain determined by the embedded position selection section is u, the 
variation of a frequency component value determined by the frequency component value variation decision section 
is AF(u) and the pseudo-random number generated by the pseudo-random number generation means is p(x), 1he 

40 pixel value variation calculation section calculates an i-th matrix element A" 1 {x t , iij) of orthogonal transform and 

calculates a variation Af(x) of a pixel value from a calculation result in accordance with a formula: 



4B 



so 
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= p(x (1> x (n) ) 

.A 1 (x (1) ,u (,> )-..,A- l (x (n) ,u (n) ) 
•AF(u (1) u { ">) 

where n is an integer equal to or higher than 1 . 

8. The digital watermark embedding device according to claim 1 or claim 2, characterized in that 

the device further comprises a digital watermark detection section (52, 72) which calculates a frequency com- 
ponent value at the watermark information embedded position based on the image into which the watermark 
information has been embedded by the pixel value conversion section, and which feeds back the frequency 
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component value or watermark information recovered from the frequency component value to the frequency 
component value variation decision section; 

the frequency component value variation decision section adaptively changes 1he variation of a frequency 
component value based on the frequency component value or the watermark information fed back by the 
s digital watermark detection section. 

9. The digital watermark embedding device according to any one of claim 1 to claim 4, characterized in that 
the device further comprises: 

JO a filtering section (53) which inputs the image into which the watermark information is embedded by the pixel 

value conversion section and which conducts a data operation estimated in advance, to the image; and 
a digital watermark detection section (52, 72) which calculates a frequency componenl value at the watermark 
information embedded position based on the image subjected to the operation of the filtering section, and 
which feeds back the frequency component value or watermark information recovered from the frequency 
component value to the frequency component value variation decision section, and wherein 
the frequency component value variation decision section adaptively changes ihe variation of a frequency 
component value based on the frequency component value or the watermark information fed back by the 
digital watermark detection means. 

so 10. The digital watermark embedding device according to any one of claim 1 to claim 4, characterized in that 

the embedded position selection section comprises: 

a frequency generation section (111) which generates a plurality of embedded position candidates; and 
a beat judgment section (112) which inputs the plurality of embedded position candidates generated by the 
25 frequency generation section, judges a degree of a beat based on relationships of frequency component values 

among the candidates, and selects and outputs a group of a predetermined number of embedded positions 
which can avoid influence of the beat, 

11. A digital watermark detection device for inputting pixel value data ol an Image into which watermark information 
30 is embedded and detecting the watermark information, characterized by comprising: 

an embedded position selection section (21, 41, 61) which determines a position in frequency domain into 
which the watermark information is embedded; 

a frequency component value calculation section (22, 42, 67) which calculates a frequency component value 
at each embedded position based on pixel value data of an image into which the watermark information has 
been embedded and on each embedded position determined by the embedded position selection section; and 
a watermark information recovery section (23, 43) which recovers the watermark information based on the 
frequency component value at each embedded position calculated by the frequency component vaiue calcu- 
lation section. 

40 

12. A digital watermark detection device for inputting pixel value data of an image into which watermark information 
is embedded using spread spectrum by ps9udo-random number series and detecting the watermark information, 
characterized by comprising: 

45 an embedded position selection section (21, 41. 61) which determines a position in frequency domain into 

which ihe watermark information is embedded; 

a pseudo-random number generation section (44) which generates a pseudo-random number series used for 
the spread spectrum; 

a frequency component value calculation section (22, 42, 67) which calculates a frequency component value 
so at each embedded position based on pixel value data of the image into which the watermark information is 

embedded, on each embedded position determined by the embedded position selection section and on the 
pseudo-random number series generated by the pseudo-random number generation section; and 
a watermark information recovery section (23, 43) which recovers the watermark information based on the 
frequency component value at each embedded position calculated by the frequency component value calcu- 
lation section. 

13. The digital watermark detection device according to claim 11, characterized in that 

while it is assumed that a pixel value at each position in pixel value domain of the image into which the 
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watermark information is embedded is f(x) and the embedded positron in frequency domain determined by the 
embedded position selection seclion is u, the frequency component value calculation section calculates an i-th 
matrix element A(u s , Xj) of orthogonal transform and calculates a frequency component value F(u) at the embedded 
position ufrom a calculation result in accordance with a formula: 

5 

AF(u t1 >,...,u (n >) 
= Z x iO)...Z x {n]A\x°\ U fl V...A-y B) , u (n) ) 
.Af(X (1) ,....X (n >) 

14. The digital watermark detection device according to claim 12, characterized in that 

while it is assumed that a pixel value at each position in pixel value domain of the image into which the 
is watermark information is embedded is f(x), the embedded position in frequency domain determined by the em- 

bedded position selection section is u and the pseudo-random number generated by the pseudo-random number 
generation means is p(Xj), ihe frequency component value calculation section calculates an i-th matrix element A 
(U;. Xi) of orthogonal transform and calculates a frequency component value F(u) at the embedded position u from 
a calculation result in accordance with a formula: 

20 

AF(u (1) u< n) ) 

= S J! (0]...S x (n)p(x (O 1 ...,x (n) ) 
-A- 1 (X t1i ,U (i) )....A- 1 (X (n) .U tn) ) 
•Aff, 0 ' ....X (n >>. 

30 15. A digital watermark embedding and detection device characterized by comprising a digital watermark embedding 
device according to claim 1 or claim 3 and a digits! watermark detection device according to claim 11 . 

16. A digital watermark embedding and detection device characterized by comprising a digital watermark embedding 
device according to claim 2 or claim 4 and a digital watermark detection device according to claim 12. 

35 

1 7. A digital watermark embedding and detection device including a digital watermark embedding function of inputting 
pixel value data of an image which is a target for embedding of watermark information and outputting, after em- 
bedding the watermark information into the image, the pixel value data of the image into which the watermark 
information has been embedded and a dighal watermark deteition function of inputting pixel value data of an image 

40 which is a target for detection of watermark information and detecting the watermark information, characterized 

by comprising: 

an embedded position seleclion section (11, 91, 61) which determines a position in frequency domain into 
■ which the watermark information Is to be embedded or the watermark information has been embedded, irre- 

as spectively of a frequency component value; 

a pseudo-random number generation means (44) which generates a pseudo-random number by which a pixel 
value at each point in pixel value domain of the image is multiplied for spread spectrum; 
a frequency component value calculation seclion (22, 42, 67) which calculates a frequency component value 
at each embedded position determined by the embedded position selection section, based on each embedded 

50 position, on a pixel value at each point in pixel value domain of the image which is the target for embedding 

of the watermark information or the image which is the target for detection of the watermark information and 
on a pseudo-random number corresponding to each point in pixel value domain of the image generated by 
the pseudo-random number generation section; 

a frequency component value variation decision section (12, 32, 202b, 512, 62) which determines a variation 
55 of a frequency component value at the each embedded position, based on the frequency component value 

calculated by the frequency component value calculation section at each embedded position determined by 
the embedded position selection sectbn; 

a pixel value variation calculation section (13, 33, 63) which calculates a variation of a pixel value al each 
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point in pixel value domain of the image which is the target for embedding ol the watermark information, based 
on the embedded position in frequency domain determined by the embedded position selection section, on 
the variation of a frequency componenl value determined by the frequency component value variation decision 
section and on the pseudo-random number corresponding to each point in pixel value domain of the image 
s generated by the pseudo-random number generation section; 

a pixel value conversion section (14, 34, 64) which changes corresponding pixel value data of the image and 
thereby embeds watermark information in accordance with the variation of a pixel value at each point in pixel 
value domain calculated by the pixel value variation calculation section; and 

a watermark information recovery section (23, 43) which recovers the watermark information based on the 
10 frequency component value at each embedded posit on of the image which is the target for detection of the 

watermark information calculated by the frequency componenl value calculation means. 

18. A digital watermark embedding device, characterized by comprising: 

15 an input section which Inputs pixel value data of an image which is a target for embedding watermark infor- 

mation; 

a frequency component pair generation section [181) which determines a pair of frequency components at 
positions in frequency domain into which the watermark information is to be embedded, irrespectively of a 
frequency component value; 

20 a pseudo-random number generation section (182) which generates a psBUdo-random number by which a 

pixel value at each point in pixel value domain of the image is multiplied for spread spectrum; 
a direct sequence spectrum spreading section (183) which outputs pixel value data spectrum-spread by mul- 
tiplying pixel value data of the image bythe pseudo-random number generated by the pseudo-random number 
generation section; 

25 an orthogonal transformation section (184) which orthogonally transforms the pixel value data spectrum- 

spread by the direct sequence spectrum spreading section and generates frequency component value data; 
a first component change section (185) which changes a frequency component value, generated by the or- 
thogonal transformation section, of the first component in the pair of frequency components at the embedded 
positions generated by the frequency component pair generation section, using a value of the watermark 

so Information; 

a second component change section (186) which changes a frequency component value, generated by the 
orthogonal transformation section, of the second component in the pair of Irequency components at the em- 
bedded positions generated by the frequency component pair generation section, using the value of the wa- 
termark information, the second component change section changing the frequency component in an opposite 
35 direction to that for 1he change by the first component change means; 

a reverse-orthogonal transformation section (187) which reverse-orthogonally Lransforms the frequency com- 
ponent value data changed bythe first component change section and the second component change section, 
and which outputs pixel value data; 

an inverse spectrum spreading section (188) which conducts inverse spectrum spreading by multiplying the 
40 - pixel value data outputted from the reverse-orthogonal transformation section by the pseudo-random number 

generated by the pseudo-random number generation section, and which outputs pixel value data of the image 
into which the watermark information has been embedded; and 

an output section which outputs the pixel value data of the image inlo which the watermark information has 
been embedded. 

45 

19. A digital watermark detection device for inputting pixel value data of an image into which watermark information 
is embedded and detecting the watermark information, characterized by comprising: 

a frequency component pair generation section (181) which determines a pair of frequency components at 
50 positions in frequency domain into which the watermark information Is to be embedded, irrespectively of a 

frequency component value; 

a pseudo-random number generation section (192) which generates a pseudo-random number by which a 
pixel value at each point in pixel value domain of the image is multiplied for spread spectrum; 
a direct sequence spectrum spreading section (183) which outputs pixel value data spectrum-spread by mul- 
55 tiplying pixel value data of the image at each point in pixel vaiue domain bythe pseudo-random number gen- 

erated by the pseudo-random number generation section; 

an orthogonal transformation section (187) which orthogonally transforms the pixel value data spectrum- 
spread by the direct sequence spectrum spreading section and generates Irequency component value data; 
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a first component average calculation section (195) which calculates an average of frequency component 
values, generated by the orthogonal transformation section, of the first component in the pairs of frequency 
components ai respective embedded positions generated bythe frequency component pair generation section; 
a second component average calculation section (1 96) which calculates an average of frequency component 
values, generated bythe orthogonal transformation section, of the second component in the pairs of frequency 
components at respective embedded positions generated bythe frequency component pair generation section; 
and 

an average comparison section (197) which compares the two averages calculated by the first component 
average calculation section and the second component average calculation section, and recovers the water- 
mark information based on the comparison result. 

20. A machine-readable recording medium which records programs for allowing a computer to function as- 

an embedded position selection section which determines a position in frequency domain into which watermark 
information is to be embedded, irrespectively of a frequency component value; 

a frequency componenl value variation decision section which determines a variation of a frequency compo- 
nent value at each embedded position determined by the embedded position selection section; 
a pixel value variation calculation section which calculates a variation of a pixel value at each point in pixel 
value domain of an image which is a target for embedding of the watermark information, based on the em- 
bedded position in frequency domain determined by the embedded position selection section and on the var- 
iation of a frequency component value determined by the frequency component value variation decision sec- 
tion; and 

a pixel value conversion section which embeds watermark information by changing corresponding pixel value 
data of the image in accordance with the variation of a pixel value at each point in pixel value domain calculated 
by the pixel value variation calculation section. 

21. A machine-readable recording medium which records programs for allowing a computer to function as: 

an embedded position selection section which determines a position in frequency domain into which watermark 
information is to be embedded, irrespectively ol a frequency component value; 

a frequency component value variation decision section which determines a variation of a frequency compo- 
nent value at each embedded position determined by the embedded position selection section; 
a pseudo-random number generation section which generates a pseudorandom number by which a pixel 
value at each point in pixel value domain of an image, which is a target for embedding of the watermark 
Information, is multiplied for spread spectrum; 

a pixel value variation calculation section which calculates a variation of a pixel value at each point in pixel 
value domain of the image based on the embedded position in frequency domain determined by the embedded 
position selection section, on the variation of a frequency component value determined by the frequency com- 
ponent value variation decision section and on a pseudo-random number corresponding to each point in fre- 
quency domain of the image generated bythe pseudo-random number generation section; and 
a pixel value conversion section which embeds watermark information by changing corresponding pixel value 
data of the imago in accordance with the variation of a pixel value ai each point in pixel value domain calculated 
by the pixel value variation calculation section. 

22. A machine-readable recording medium which records programs for allowing a computer to function as: 

an embedded position selection section which dete-mines a position in frequency domain into which watermark 
information is to be embedded, irrespectively of a frequency component value; 

a frequency component value calculation section which calculates a frequency component value at Ihe em- 
bedded position determined by the embedded position selection section based on a pixel value at each point 
in pixel value domain of an Image which is a target for embedding of the watermark information; 
a frequency component value variation decision section which deteimines a variation of a frequency compo- 
nent value at each embedded position determined by the embedded position selection section based on the 
frequency component value at the embedded position calculated by the frequency component value calcula- 
tion section; 

a pixel vafue variation calculation section which calculates a variation of a pixel value at each point in pixel 
value domain ol the image based on the embedded position in frequency domain determined by the embedded 
position selection section and on the variation of a frequency component value determined by the frequency 
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component value variation decision section; and 

a pixel value conversion section which embeds watermark information by changing corresponding pixel value 
data of the image in accordance with the variation of a pixel value at each point in pixel value domain calculated 
by the pixel value variation calculation section. 

23. A machine-readable recording medium which records programs for allowing a computer to function as: 

an embedded position selection section which determines a position in f requency domain into which watermark 
information is to be embedded, irrespectively of a frequency component value; 

a pseudo-random number generation section which generates a pseudo-random number by which a pixel 
value at each point in pixel value domain of an image, which is a target for embedding of 1he watermark 
Information, is multiplied tor spread spectrum; 

a frequency component value calculation section which calculates a frequency component value at each em- 
bedded position determined by the embedded position selection section based on the embedded position, a 
pixel value a! each point in pixel value domain of the image and a pseudo-random number corresponding to 
each point in pixel value domain of the image generated by the pseudo-random number generation section; • 
a frequency component value variation decision' section which determines a variation of a frequency compo- 
nent value at each embedded position determined by the embedded position selection section based on the 
frequency component value calculated by the frequency component value calculation section, at each em- 
bedded position determined by the embedded position selection section; 

a pixel value variation calculation section which calculates a variation of a pixel value of the image at each 
point in pixel value domain based on the embedded position in frequency domain determined by the embedded 
position selectbn section, on the variation of a frequency component value determined by the frequency com- 
ponent value variation decision section and a pseudo-random 'number generated by the pseudo-random 
number generation section and corresponding to each point in frequency domain of the image; and 
a pixel value conversion section which embeds watermark information by changing corresponding pixel value 
data of the image in accordance with the variation of a pixel value at each point in pixel value domain calculated 
by the pixel value variation calculation section. 

24. A machine-readable recording medium which records programs for allowing a computerto function as: 

an embedded position selection section which determinesaposition inf requency domain into which watermark 
information is embedded; 

a frequency component value calculation section which calculates a frequency component value at each em- 
bedded position based on pixel value data of an image into which the watermark information has been em- 
bedded and on each embedded position determined by the embedded position selection section, and 
a watermark information recovery section which recovers the watermark information based on the frequency 
component value at each embedded position calculated by the frequency component value calculation section. 

25. A machine-readable recording medium which records programs for allowing a computer to function as: 

an embedded position selection section which detemninesa position inf requency domain into which watermark 
information is embedded by using spread spectrum by means of pseudo-random number series; 
a pseudo-random number generation section which generates a pseudo-random number series used for the 
spread spectrum; 

a frequency component value calculation section which calculates a frequency component value at each em- 
bedded position based on pixel value data of the image into which the watermark information is embedded, 
on each embedded position determined by the embedded position seleclion section and on the pseudo-random 
number series generated by the pseudo-random number generation section; and 

a watermark information recovery section which recovers the watermark information based on the frequency 
component value at each embedded position calculated by the frequency component value calculation section. 
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